study 
Report 
2007-  01 


Immersive  Simulation  Training 
for  the  Dismounted  Soldier 


Bruce  W.  Knerr 

U.S.  Army  Research  institute 


2007031 941 4 


United  States  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences 


February  2007 


Approved  for  public  release;  distribution  is  unlimited. 


U.S.  Army  Research  Institute 

for  the  Behavioral  and  Social  Sciences 


A  Directorate  of  the  Department  of  the  Army 
Deputy  Chief  of  Staff,  G1 

Authorized  and  approved  for  distribution: 

SCOTT  E  GRAHAM  MICHELLE  SAMS 

Acting  Technicai  Director  Acting  Director 


Technical  review  by 

Timothy  Sayers,  Army’s  Combat  Developer  Project  Lead 
for  Dismounted  Soldier  Simulations 

Diana  Tierney,  Chief,  S&T  Research,  Studies  and  Experimentation 
Division,  TPCRD,  DCSOPS&T,  TRADOC 


NOTICES 

DISTRIBUTION:  Primary  distribution  of  this  Study  Report  has  been  made  by  ARI. 
Please  address  correspondence  concerning  distribution  of  reports  to:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences,  Attn:  DAPE-ARI-MS, 

2511  Jefferson  Davis  Highway,  Arlington,  Virginia  22202-3926. 

FINAL  DISPOSITION:  This  Study  Report  may  be  destroyed  when  it  is  no  longer 
needed.  Please  do  not  return  it  to  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences. 

NOTE:  The  findings  in  this  Study  Report  are  not  to  be  construed  as  an  official 
Department  of  the  Army  position,  unless  so  designated  by  other  authorized  documents. 


REPORT  DOCUMENTATION  PAGE 


1.  REPORT  DATE  (dd-mm-yy)  2.  REPORT  TYPE 

February  2007  Final 


4.  TITLE  AND  SUBTITLE 

Immersive  Simulation  Training  for  the  Dismounted  Soldier 


6.  AUTHOR(S) 


Bruce  W.  Knerr 


3.  DATES  COVERED  (from.  . .  to) 

October  2005  -  October  2006 


5a.  CONTRACT  OR  GRANT  NUMBER 


5b.  PROGRAM  ELEMENT  NUMBER 

665803 


5c.  PROJECT  NUMBER 

D730 


5d.  TASK  NUMBER 

338 


5e.  WORK  UNIT  NUMBER 

H01 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES)  8.  PERFORMING  ORGANIZATION  REPORT  NUMBER 

U.  S.  Army  Research  Institute  for  the  Behavioral  &  Social 
Sciences 

ATTN:  DAPE-ARI-IF 
12350  Research  Parkway 
Orlando,  FL  32826-3276 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

U.  S.  Army  Research  Institute  for  the  Behavioral  &  Social 
Sciences 

2511  Jefferson  Davis  Highway 
Arlington,  VA  22202-3926 


12.  DISTRIBUTION/AVAILjABILITY  statement 

Approved  for  public  release;  distribution  is  unlimited. 


13.  SUPPLEMENTARY  NOTES 

Subject  Matter  POO:  Bruce  W.  Knerr 


14.  ABSTRACT  (Maximum  200  words): 

A  study  was  conducted  to  document  the  need  for  immersive  dismounted  virtual  Soldier  and  leader  training 
and  the  available  research  evidence  regarding  the  effectiveness  of  virtual  training  for  training  Soldiers  and 
leaders  in  complex  skills.  A  literature  search  of  research  reports,  journal  articles,  and  conference 
proceedings  to  identify  evaluations  and  experiments  related  to  the  study  topic  of  the  training  effectiveness 
of  immersive  virtual  simulations  was  conducted.  Particular  attention  was  paid  to  an  expended  series  of 
evaluations  conducted  by  the  Army  R&D  organizations  during  the  period  1997  -  2005.  The  major  findings 
are  organized  around  the  topics  of  training  effectiveness.  Soldier  task  performance,  and  advantages  and 
disadvantages  of  immersive  virtual  simulations.  Soldiers  and  small  unit  leaders  report  that  their  skills 
improve  as  a  result  of  training  in  virtual  simulations,  and  these  self-reports  by  have  generally,  if  informally, 
been  confirmed  by  observers.  While  the  simulators  impose  constraints  on  the  performance  of  some 
Soldier  activities,  this  should  limit  training  effectiveness  only  if  those  activities  that  cannot  be  performed  in 
the  simulator  are  not  trained  by  other  means.  Advantages  and  disadvantages  of  immersive  simulations 
are  also  described. 


10.  MONITOR  ACRONYM 


11.  MONITOR  REPORT  NUMBER 

Study  Report  2007-01 


15.  SUBJECT  TERMS 

Immersive  simulation,  virtual  simulation,  virtual  environment,  mission  rehearsal,  collective  training.  Infantry 
training 


21.  RESPONSIBLE  PERSON 
Ellen  Kinzer 

Technical  Publications  Specialist 
703-602-8047 


SECURITY  CLASSIFICATION  OF 

16.  REPORT 

Unclassified 

17.  ABSTRACT 

Unclassified 

18.  THIS  PAGE 

Unclassified 

19.  LIMITATION  OF  20.  NUMBER 


ABSTRACT 


OF  PAGES 


11 


study  Report  2007-01 


Immersive  Simulation  Training  for  the  Dismounted  Soldier 


Bruce  W.  Knerr 

U.S.  Army  Research  Institute 


Simulator  Systems  Research  Unit 
Stephen  L.  Goldberg,  Chief 


U.S,  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
2511  Jefferson  Davis  Highway,  Arlington,  Virginia  22202-3926 


February  2007 


Army  Project  Number  Personnel  and  Training 

665803D730  Analysis  Activities 

Approved  for  public  release;  distribution  is  unlimited. 


iii 


IV 


IMMERSIVE  SIMULATION  TRAINING  FOR  THE  DISMOUNTED  SOLDIER 

EXECUTIVE  SUMMARY 


Research  Requirement; 

ARI  was  tasked  to  conduct  a  study  to  develop  a  training/benefit  analysis  documenting  the 
need  for  and  expected  benefits  of  immersive  dismounted  virtual  Soldier  and  leader  training,  and 
to  prepare  a  report  documenting  the  available  research  evidence  regarding  the  effectiveness  of 
virtual  training  for  training  Soldiers  and  leaders  in  complex  skills,  the  cognitive  and  decision 
making  skills  that  will  foster  adaptability  and  the  capability  to  respond  to  rapidly  changing 
situations. 

Procedure: 

ARI  conducted  a  literature  search  of  research  reports  in  the  Defense  Technical 
Information  Center  (DTIC),  and  journal  articles  and  conference  proceedings  in  the  open 
literature  to  identify  evaluations  and  experiments  related  to  the  study  topic  of  the  training 
effectiveness  of  immersive  virtual  simulations.  Particular  attention  was  paid  to  an  expended 
series  of  evaluations  conducted  by  the  U.S.  Army  Research  Institute  (ARI),  Program  Executive 
Office,  Simulation,  Training  and  Instrumentation  (PEO  STRI),  the  Army  Research  Laboratory 
(ARL),  and  the  Research,  Development,  and  Engineering  Command  Simulation  and  Training 
Technology  Center  (RDECOM  STTC)  during  the  period  1997  -  2005.  This  series  of  related 
development  and  evaluation  activities  constitutes  the  most  extensive  examination  of  questions  of 
dismounted  Soldier  performance  and  training  effectiveness  in  immersive  simulations. 

Findings: 

It  is  difficult  to  draw  definitive  conclusions  about  the  effectiveness  of  immersive  virtual 
simulation  for  training  dismounted  Soldiers  and  leaders  because  the  technology  is  relatively  new, 
and  organizations  have  not  had  the  resources  to  conduct  evaluations  of  sufficient  scope  to 
provide  definitive  results  about  training  effectiveness.  The  evidence  that  is  available  comes  from 
assessments  involving  small  numbers  of  Soldiers  and  units  who  received  limited  amounts  of 
training.  In  these  assessments.  Soldiers  and  small  unit  leaders  have  consistently  reported  that 
their  skills  improved  as  a  result  of  training  in  virtual  simulations,  with  the  most  improvement  in 
controlling,  coordinating,  communicating,  and  planning  (conducted  prior  to  use  of  the 
simulators),  and  less  improvement  in  the  mechanics  of  tasks  and  battle  drills.  These  self-reports 
have  generally  been  confirmed  by  observers.  They  are  also  consistent  with  evaluations  of 
collective  marksmanship  trainers,  which  use  different  technology,  but  are  functionally  similar  to 
immersive  virtual  simulators.  There  is  objective  performance  data  indicating  that  the 
marksmanship  training  provide  effective  training  in  tactical  skills  as  well  as  marksmanship. 

Virtual  immersive  simulators  necessarily  do  limit  how  some  tasks  can  be  performed, 
particularly  those  involving  locomotion  and  touch.  This  limits  training  effectiveness  only  if 
those  actions  that  cannot  be  performed  in  the  simulator  are  not  trained  by  other  means.  Even 
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though  the  simulator  may  not  allow  Soldiers  to  perform  physieal  actions  as  they  do  in  the  real 
world,  it  does  allow  them  to  practice  and  learn  the  cognitive  skills  they  need.  These  include  the 
planning  that  needs  to  be  done  prior  to  missions,  the  skills  required  to  maintain  situational 
awareness,  and  learning  to  think  on  their  feet  and  make  appropriate  decisions  in  complex 
situations. 

Virtual  simulations  possess  a  number  of  advantages  relative  to  the  use  of  live  simulation 
for  dismounted  Soldier  training.  They  can  represent  any  terrain  or  environment  that  has  been 
modeled,  weather,  and  other  environmental  conditions.  Use  of  immersive  virtual  simulation  is 
likely  to  reduce  the  amount  of  time  required  by  a  unit  to  conduct  a  training  exercise  relative  to 
live  training.  Fewer  human  role  players  should  be  required  to  fill  the  positions  of  enemy  units, 
other  friendly  units,  and  civilians  in  virtual  simulations  than  in  live  simulations.  The  reduced 
risk  of  accident  or  injury  makes  it  possible  to  more  realistically  depict  the  effects  of  demolitions 
or  indirect  fire  than  is  possible  in  live  simulations.  Virtual  simulations  require  less  physical 
space  than  a  live  simulation  facility,  and  reduce  the  need  for  use  of  actual  equipment. 

Virtual  After  Action  Reviews  (AARs)  provide  unique  capabilities  that  are  not  currently 
possible  with  live  simulations.  They  can  provide  an  overall  view  of  the  action  that  is  not 
possible  with  video  cameras,  which  provide  multiple  separate  views  of  portions  of  the  action. 
They  can  provide  a  view  of  the  action  from  any  perspective,  not  limited  by  camera  locations. 
They  capture  and  provide  an  accurate  representation  of  ground  truth.  Finally,  they  can  provide  a 
variety  of  aids  to  analyze  and  depict  events. 

Virtual  simulations  possess  a  number  of  disadvantages  relative  to  the  use  of  live 
simulation  for  dismounted  Soldier  training.  Some  physical  activities  cannot  be  practiced. 
Weapons  effects  may  not  be  modeled  correctly,  or  position  and  location  tracking  may  not  be 
accurate  enough  to  depict  realistic  weapons  effects.  Soldiers  may,  therefore,  develop 
inappropriate  expectations.  Virtual  simulations  require  improved  facilities  (enclosed  space,  air 
conditioning,  and  power)  and  technical  support  to  implement  training  scenarios  and  operate  and 
maintain  computers  and  other  equipment.  Trainees  may  not  take  virtual  simulations  seriously 
because  they  are  similar  to  video  games,  and  might,  therefore,  practice  actions  that  are 
inappropriate  in  the  real  world,  or  attempt  to  “game”  the  system.  Some  trainees  may  experience 
“simulator  sickness.” 

Virtual  immersive  simulations  can  provide  the  opportunity  for  small  unit  leaders  and  unit 
members  to  practice,  in  the  context  of  realistic  simulated  operations,  the  cognitive  and  decision 
making  skills  that  will  foster  adaptability  and  the  capability  to  respond  to  rapidly  changing 
situations.  Focused,  repetitive,  deliberate  practice,  with  feedback  based  on  performance,  is  an 
effective  method  for  training  the  recognition  of  situations  and  developing  expertise.  Cognitive 
and  decision  making  skills  can  be  trained  even  if  some  physical  tasks  cannot  be  performed  in  the 
situation. 

Utilization  and  Dissemination  of  Findings 

It  is  recommended  that  the  following  actions  be  taken  as  the  next  steps  toward  acquisition 
and  fielding  of  an  immersive  virtual  training  system  for  dismounted  Soldiers. 
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Conduct  a  large-scale  evaluation  using  current  technology.  This  evaluation  should  consider 
objective  measurement  of  both  skill  improvement  on  the  simulator  and  transfer  of  those  skills  to 
a  live  simulation  environment.  A  sufficient  number  of  units  should  participate  so  that  any 
meaningful  real  differences  in  the  training  effectiveness  of  different  can  be  detected.  The 
evaluation  should  be  embedded  in  the  unit’s  normal  training  progression,  and  unit  personnel 
should  be  involved  in  its  development  and  delivery. 

Consider  cost-effectiveness  of  fully  immersive  and  desktop  or  laptop  systems.  There  is  a  large 
cost  differential  between  fiilly-immersive  simulators  and  simulators  using  desktop  or  laptop 
computers  to  provide  the  same  functionality.  In  contrast,  the  research  evidence  indicates  that  any 
difference  in  training  effectiveness  between  immersive  and  desktop  systems  is  likely  to  be  small, 
although  immersive  systems  have  a  small  advantage  when  training  spatially-oriented  tasks. 
Whether  they  are  better  when  training  squads  to  conduct  urban  or  counter-insurgency  operations, 
and  if  so,  whether  the  difference  is  large  enough  to  justify  the  increased  cost,  are  unknown,  but 
should  be  investigated. 

Conduct  a  survey  of  potential  officer  and  enlisted  trainees  to  determine  game-playing  skill.  An 
assumption  underlying  the  growing  popularity  of  the  use  of  video  and  PC-based  games  for 
training  is  that  the  potential  trainees  are  interested  in  and  experienced  with  such  games.  They, 
therefore,  need  little  training  in  their  use,  and  are  motivated  to  play.  However,  data  from  small 
samples  of  Soldiers  who  participated  in  evaluations  suggests  that  this  may  not  be  the  case.  Since 
game  playing  experience  has  implications  for  the  types  of  games  that  could  be  used  for  training 
and  the  amount  and  type  of  prerequisite  game  playing  training  required,  we  would  like  to  have  a 
much  better  picture  of  the  both  current  and  projected  Soldier  and  leader  familiarity  and 
experience  with  different  types  of  interactive  games. 

Continue  to  improve  virtual  simulation  capabilities  and  assess  their  training  effectiveness.  Some 
aspects  of  immersive  virtual  simulation  technology,  such  as  large-screen  displays,  will  continue 
to  improve,  or  become  less  expensive,  as  a  result  of  commercial  market  forces.  However,  it  will 
require  a  continuing  science  and  technology  investment  by  the  Army  to  produce  a  tailored 
system  that  provides  cost-effective  training. 
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Introduction 

TRADOC  DCSOPS&T  and  TPIO  Virtual  requested  that  ARI  conduct  a  study  to  develop 
a  training/benefit  analysis  documenting  the  need  for  and  expected  benefits  of  immersive 
dismounted  virtual  Soldier  and  leader  training.  This  report  documents  the  available  research 
evidence  regarding  the  effectiveness  of  virtual  training  for  training  Soldiers  and  leaders  in 
complex  skills.  It  is  based  on  an  extensive  review  of  research  findings  Ifom  all  Services  and 
industry  that  impact  upon  the  use  of  immersive  simulation  for  the  dismounted  Soldier. 

Study  Objectives 

The  objectives  of  this  study  are:  (a)  to  review  and  synthesize  the  accumulated  evidence, 
from  the  research  literature  and  reports  of  field  experiments,  that  is  related  to  the  effectiveness  of 
virtual  simulation  for  dismounted  Soldier  training;  (b)  to  use  this  information  as  a  basis  for 
conclusions  about  the  expected  effectiveness  of  virtual  training  approaches;  and  (c)  to  present 
this  information  in  a  straightforward,  non-technical  format. 

To  accomplish  those  objectives,  this  report  is  organized  around  fiver  major  questions. 

•  What  is  the  Army’s  training  need  which  is  to  be  met  through  the  application  of  virtual 
simulation  and  gaming  technology? 

•  What  virtual  simulation  and  gaming  technology  exists  today  to  meet  this  need? 

•  What  evidence  is  there  that  virtual  simulation  and  gaming  technology  can  meet  those 
training  needs? 

•  What  recommendations  for  future  actions  can  be  made  based  on  this  evidence? 

•  What  are  the  unresolved  issues  and  further  research/study  needs? 


Immersive  Simulations  for  Dismounted  Soldiers:  The  Army  Training  Need 

This  report  will  not  provide  a  detailed  or  exhaustive  task-level  analysis  of  the  specific 
skills  that  immersive  virtual  simulation  is  expected  to  train.  However,  the  Army  Strategic 
Planning  Guidance  2005  (Department  of  the  Army,  2005)  and  the  Soldier  CATT  Operational 
Requirements  Document  (ORD;  Department  of  the  Army,  2004)  provide  a  broad  picture  of 
requirements.  Three  major  needs  emerge.  First,  Soldiers  and  small  unit  leaders  need  to  be 
adaptable  and  capable  of  responding  to  and  taking  advantage  of  rapidly  changing  situations. 
Second,  they  must  be  attuned  to  cultural  conditions  and  knowledgeable  in  the  culture,  history, 
and  language  of  the  area  of  operations.  Third,  they  must  be  able  to  rapidly  and  accurately  assess 
the  evolving  situation  using  all  of  the  tools  at  their  disposal.  Officer  and  junior  non¬ 
commissioned  officers  must  also  be  able  to  communicate  effectively  with  a  variety  of  different 
governmental  and  non-governmental  organizations,  to  include  media  representatives.  Table  1 
provides  more  detail  about  these  needs. 
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Capabilities  to  be  provided  by  Immersive  Simulation 

The  Soldier  CATT  ORD  also  deseribes  the  eapabilities  to  be  provided  by  immersive 
simulation.  The  following  exeerpts  provide  a  high  level  summary  of  the  capabilities. 

[The]  centerpiece  is  the  “Virtual  Warrior,”  a  high  fidelity  dismounted  leader 
trainer  providing  a  natural  virtual  environment  in  which  dismounted  leaders  see 
the  battlefield  in  three  dimensions,  control  subordinate  virtual  Soldiers  through 
voice  recognition/voice  synthesis,  and  communicate  via  FM  voice 
communications,  (p.  6) 

Soldier  CATT  is  a  Soldier  and  leader  collective  combined  arms  virtual  training 
simulation  focused  on  dismounted  infantry  centric  training.  It  will  provide  a 
sufficient  quantity  of  Virtual  Warrior  simulators  to  support  from  individual 
Soldier  training  at  the  squad  level  up  to  platoon  and  infantry  company  leader 
training.  Soldier  CATT  is  designed  to  train  ...  the  full-spectrum  of  operations  in 
a  combined  arms  and  joint  environment.  Leaders  will  train  such  tasks  as  the 
command  and  control  of  units,  fire  coordination,  employment  of  joint  enablers, 
engineer  support  (obstacle  integration)  and  logistical  support.  Soldier  CATT  will 
support  full  spectrum  training  (e.g.,  infantry  tasks  of  raid,  dismounted  assault  and 
area  presence;  Reconnaissance,  Surveillance,  and  Target  Acquisition  (RSTA) 
tasks  of  multidimensional  area  reconnaissance,  multidimensional  route 
reconnaissance  and  screen).  Additionally,  Soldier  CATT  will  support  training  for 
urban  operations  (UO)  and  operations  on  complex  terrain,  (p.  7) 

Soldier  CATT  will  provide  ...  a  home  station  and  deployed  unit  training 
capability  ideally  suited  for  walk-level  and  sustainment  training  as  well  as 
mission  rehearsal  training,  (p.  7) 
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Table  1 

Training  Need  Statements 


Recent  experience  in  Iraq  and  Afghanistan  has  shown  the  need  for  Soldiers  who  are  not  only  well  trained  and 
equipped,  but  who  are  also  adaptable  and  capable  of  responding  to  rapidly  changing  situations  and  are  attuned  to 
cultural  conditions.  In  addition  to  rigorous,  repetitive  training  in  weapons  and  fieldcraft.  Soldiers  also  require 
training  in  how  to  adapt  to  quickly  evolving  situations,  not  just  how  to  react  to  changes,  but  also  how  to  shape  the 
environment  to  create  the  best  possible  outcomes,  (Army  Strategic  Planning  Guidance  2005,  p.  4) 

Iraq  in  particular  has  proven  to  be  a  non-linear  battlefield.  Given  the  security  environment,  we  can  expect  this  to 
become  the  norm.  This  requires  all  Soldiers  understand  they  are  warriors  first  and  that  they  must  be  ready  to  fight. 
The  distinctions  between  combatant  and  noncombatant  have  blurred,  as  have  the  distinctions  between  combat 
operations  and  stability  operations.  Simultaneous  operations  across  the  range  of  military  operations,  rather  than 
sequential  operations  will  likely  be  the  rule.  To  succeed  in  such  an  environment,  Soldiers  and  leaders  must  be 
capable  of  rapidly  and  accurately  assessing  the  evolving  situation.  In  order  to  make  accurate  assessments,  they  must 
also  be  capable  of  using  all  the  tools  at  their  disposal.  This  requires  Soldiers  and  leaders  who  understand  and  are 
capable  of  leveraging  the  latest  technology  and  Joint  Capabilities.  It  also  requires  Soldiers  and  leaders  who  are 
knowledgeable  of  culture,  history  and  the  language  of  the  area  of  operations.  (Army  Strategic  Planning  Guidance 
2005,  p.  5) 

Today’s  environment  requires  lieutenants  and  junior  noncommissioned  officers  to  communicate  regularly  with  a 
variety  of  actors;  from  interagency  partners,  to  members  of  non-governmental  organizations,  to  local  leaders,  to 
members  of  the  U.S.  and  foreign  media.  To  succeed,  leaders  at  all  levels  must  have  situational  understanding  that 
extends  beyond  the  tactical  level.  This  requires  a  robust  leader  development  system  that  grows  leaders  who  are 
prepared,  versatile  and  adaptive  and  who  have  a  broad  understanding  of  the  political  and  military  objectives  of  the 
campaign,  as  well  as  the  potential  implications  of  their  actions  upon  those  objectives.  (Army  Strategic  Planning 
Guidance  2005,  pp.  5-6) 

Recent  operations,  such  as  operations  in  Afghanistan  and  Iraq,  identified  an  immediate  and  critical  need  for  a  virtual 
training  simulation  to  build  and  sustain  the  training  readiness  of  dismounted  infantry-centric  units  . . .  required  to 
conduct  close  combat  operations  in  complex/restrictive  terrain  against  asymmetric  forces,  integrating  multiple 
enablers  and  platforms  and  the  associated  fires  and  effects  of  fires  in  the  joint  and/or  multinational  operational 
battlespace.  (Soldier  CATT  ORD  (revised  8  June  2004),  p.  1). 

Virtual  training  is  required  to  prepare  and  build  eonfident  and  adaptive  dismounted  infantry  leaders  and  Soldiers, 
that  can  rapidly  dominate  battlefield  situations,  and  dilemmas  in  varying  conditions  (combined  arms,  joint 
interagency  multi-national,  contemporary  operational  environment,  urban  and  complex  terrain),  through  small  unit 
actions,  while  dispersed,  deployed,  or  distributed,  and  remain  self  aware  while  digitally  connected  or  networked 
between  shooters  and  sensors,  and  dismounted  and  mounted  forces.  Since  every  Soldier  must  be  prepared  to  be  a 
leader,  virtual  training  will  improve  the  dismounted  Soldier’s  ability  to  remotely  control  weapon  and  sensor  systems 
to  include  robotic  systems  and  unmarmed  platforms  to  enhance  his  ability  to  see,  understand,  and  act  first.  (Soldier 
CATT  ORD  (revised  8  June  2004),  p.  1) 

The  dismounted  Soldier  and  leader  require  training  in  a  multiplicity  of  environments  and  terrain  (e.g.  urban,  desert, 
mountainous,  jungle)  that  generally  cannot  be  replicated  at  homestation  training  areas  or  the  Combat  Training 
Centers.  (Soldier  CATT  ORD  (revised  8  June  2004),  p.  2) 

It  is  a  critical  requirement  to  prepare  junior  leaders  to  successfully  lead  multifunctional  units,  effectively  use 
digitized  systems,  integrate  into  combined  arms  and  joint  organization  and  conduct  decentralized  and  dispersed 
operations  across  the  fiill-spectrum  of  possible  missions.  To  develop  and  sustain  the  taetical  and  technical 
proficiency  required  of  current  and  future  Soldiers  and  leaders,  units  need  home  station  and  deployable  training  tools 
that  enable  frequent  and  repetitious  complex  task  training.  (Soldier  CATT  ORD  (revised  8  June  2004),  p.  2) 
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The  Soldier  CATT  ORD  provides  a  more  detailed  description  of  the  training 

requirements  that  a  virtual  simulation  system  should  satisfy: 

•  General  Requirements: 

o  Support  training  for  dismounted  infantry  from  squad  to  company  level, 
o  Allow  the  expansion  of  the  training  audience  from  individual  Soldier  up  to  battalion 
level  leader. 

o  Support  the  mounted  training  requirements  of  the  Stryker  Brigade  Combat  Team, 
o  Support  initial  leader  training  at  institutions  (USAIS,  USJFKSWCS,  and  the 
USAARMC). 

o  Support  unit  sustainment  training. 

o  Provide  structured,  multi-echelon,  collective  task  training  to  active  and  reserve 
component  dismounted  infantry  units. 

o  Support  Soldier  and  leader  training  during  peacetime,  mobilization  and  deployment, 
o  Support  conduct  of  command  and  control  and  tactical  training, 
o  Support  external  evaluations,  mission  training  and  combined  arms  exercises, 
o  Complement  live  training. 

o  Allow  dismounted  leaders,  mounted  leaders  and  crews,  and  units  to  operate  in 
combined  L-V-C  exercises. 

•  Institutional  Training  Requirements: 

o  Enhance  leader  skill  training  in  subjects  including  decision  making,  planning  for 
operations,  command  and  control  of  small  units,  direct  fire  coordination,  fire  support 
coordination,  integration  of  obstacles  and  logistical  support, 
o  Incorporated  into  programs  of  instruction  for  infantry,  SOF  and  scout  officer  and 
enlisted  leader  courses. 

•  Unit  Training  Requirements: 

o  Support  TOE  unit  training  requirements  for  active  and  reserve  component  training  at 
home  station  and  in  deployed  locations  worldwide, 
o  Teach  and  sustain  proficiency  in  individual  and  collective  leader  tasks, 
o  Live  force-on-force  exercises  required  to  validate  proficiency. 


What  is  Virtual  Simulation? 

Virtual  simulation  is  defined  in  TRADOC  Pamphlet  350-37  as  “a  simulation  involving 
real  people  operating  simulated  systems.  Virtual  simulations  inject  humans-in-the-loop  in  a 
central  role,  by  exercising  motor  skills,  decision  skills,  or  communication  skills.”  (TRADOC 
Pamphlet  350-37,  9  June  2003) 

The  Columbia  Electronic  Encyclopedia  provides  a  more  detailed  and  more  descriptive 
definition  of  largely  equivalent  and  interchangeable  terms  virtual  reality  (VR)  or  virtual 
environment  (VE).  “Computer-generated  environment  with  and  within  which  people  can 
interact.  The  advantage  of  VR  is  that  it  can  immerse  people  in  an  environment  that  would 
nonnally  be  unavailable  due  to  cost,  safety,  or  perception  restrictions.  A  successful  VR 
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environment  offers  users  immersion,  navigation,  and  manipulation.  VR  encompasses  a  range  of 
interactive  computer  environments,  from  text-oriented  on-line  forums  and  multiplayer  games  to 
complex  simulations  that  combine  audio;  video,  animation,  or  three-dimensional  graphics;  and 
scent.  Some  of  the  more  realistic  effects  are  achieved  using  a  helmet  like  apparatus  with  tiny 
computer  screens,  one  in  front  of  each  eye  and  each  giving  a  slightly  different  view  so  as  to 
mimic  stereoscopic  vision.  Sensors  attached  to  the  participant  (e.g.,  gloves,  bodysuit,  footwear) 
pass  on  his  or  her  movements  to  the  computer,  which  changes  the  graphics  accordingly  to  give 
the  participant  the  feeling  of  movement  through  the  scene.  Computer-generated  physical 
feedback  adds  a  “feel”  to  the  visual  illusion,  and  computer-controlled  sounds  and  odors  reinforce 
the  virtual  environment.  Other  VR  systems,  such  as  flight  simulators,  use  larger  displays  and 
enclosed  environments  to  create  an  illusion.  Less-complicated  systems  for  personal  computers 
manipulate  an  image  of  three-dimensional  space  on  a  computer  screen.  In  a  virtual  network 
many  users  can  be  immersed  in  the  same  simulation,  each  perceiving  it  from  a  personal  point  of 
view.  VR  is  used  in  some  electronic  games,  in  amusement-park  attractions,  in  military  exercises, 
and  to  simulate  construction  designs.  Experimental  and  envisioned  uses  include  education, 
industrial  design,  surgical  training,  and  art.”  (virtual  reality,  2003) 

For  purposes  of  this  report,  immersive  simulations  are  defined  to  include  a  broad  range  of 
virtual  simulations.  At  one  extreme,  it  will  include  the  use  of  videogame  platforms  (e.g..  Full 
Spectrum  Warrior).  At  the  other  extreme,  it  will  include  networked  fully  immersive  simulators 
with  full-screen  or  head-mounted  displays  (e.g.,  the  Virtual  Integrated  MOUT  Training  System 
(V-IMTS).  Between  these  two  extremes,  a  variety  of  PC-based  simulations  are  possible  and  will 
be  considered. 

Campbell,  Knerr,  and  Lampton  (2004)  have  provided  a  list  of  necessary  functions  that 
must  be  performed  in  order  to  use  virtual  simulations  for  dismounted  Soldier  training. 

•  Realistic  representation  of  terrain  in  different  conditions  (e.g.,  night,  fog)  -  How  will  the 
Soldier  see  his  environment? 

•  Locomotion  devices  that  provide  realistic  perception  of  movement  and  some  degree  of 
accurate  energy  expenditure  -  How  can  he  move  or  change  position?  How  can  he  cross  a 
field  or  climb  stairs? 

•  Ability  to  select  weapons  and  other  systems  and  use  them  with  realistic  effects  -  How  will 
the  Soldier  be  able  to  use  his  M4  rifle,  throw  smoke  grenades  or  concussion  grenades,  and 
use  binoculars  or  night  vision  devices? 

•  “Intelligent”  computer-controlled  forces  to  represent  enemy,  friendly,  and  neutral  forces  - 
How  can  the  simulated  people  react  to  the  Soldier 's  actions?  Will  they  fall  when  shot,  and 
walk  around  rather  than  walk  into  Soldiers  and  other  obstacles? 

•  Dynamic  terrain  showing  damage  to  structures,  rubble  and  other  micro-terrain  obstacles  - 
How  can  scenery  features  (such  as  trees,  cars,  and  street  lights)  also  "behave  ”  like  their  real 
world  counterparts?  Will  buildings  and  other  structures  show  effects  of  the  Soldier 's 
actions? 
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•  Accurate  portrayal  of  Soldier  actions  and  movements  in  the  virtual  environment  -How  do  the 
Soldiers  see  each  other  or  interact? 

•  Voice  and  gesture  recognition  by  simulated  “Soldiers”  -  How  can  leader  trainees  control 
computer-generated  Soldiers  with  voice  and  hand-arm  signals? 

•  Performance  recording,  analysis,  and  tailorable  feedback  at  an  AAR  station  -  How  can  a 
trainer  know  what’s  going  on  in  the  simulation,  so  there  can  be  appropriate  feedback  and 
performance  improvement? 

While  there  are  multiple  ways  the  essential  components  and  capabilities  required  for 

dismounted  Soldier  simulation  could  be  configured  or  described,  the  following  categorization 

covers  the  key  elements. 

•  Computer  environment  generator/processor.  This  component  displays  the  simulated 
environment  and  processes  calculations  to  determine  environments  and  weapons  effects.  It  is 
usually  a  PC,  although  it  may  also  be  a  game  console. 

•  Network.  This  transmits  information  among  the  participating  simulators  and  between  the 
simulators  and  the  management  system.  Network  characteristics  depend  on  the  simulation. 
High-end  military  systems  typically  use  a  Distributed  Interactive  Simulation  (DIS)  or  High 
Level  Architecture  (HLA)  network.  Systems  based  on  commercial  game  engines  use 
protocols  developed  for  use  over  the  internet. 

•  Trainee  Control  Suite.  This  consists  of  the  components  of  the  individual  simulator  by  which 
the  trainee  affects  the  simulation.  The  major  components  are: 

o  Weapon  Controller  -  A  deviee  for  controlling  the  trainee’s  weapon  (aim  point, 
ammunition  selection,  firing  rate,  etc.).  It  could  be  as  sophistieated  as  an 
instrumented,  demilitarized  weapon,  or  as  simple  as  a  game  controller. 

o  Locomotion  Controller  -  A  device  for  controlling  the  trainee’s  posture, 

orientation,  and  movement.  The  range  of  sophistication  is  quite  varied.  It  could  be 
combined  with  the  Weapon  Controller. 

o  Voice  Input  device  -  A  microphone  is  used  to  capture  the  voice  signal,  but  there 
are  many  ways  to  process  and  transmit  it. 

o  Other  Devices  -  To  select  or  control  additional  equipment  such  as  digital  systems, 
night  vision  goggles,  etc. 

•  Trainee  Display  Suite.  This  consists  of  the  components  of  the  individual  simulator  which 
present  infonnation  to  the  trainee.  The  major  components  are: 

o  Visual  Display  -  There  are  three  approaches: 

■  Large-screen  displays.  These  have  typically  been  rear-sereen  projection 
displays,  although  some  systems  have  used  front-projeetion  displays.  They 
may  be  either  single-screen  or  multi-screen.  Large  Light  Emitting  Diode 
(LED)  and  plasma  displays  may  become  a  reasonable  alternative. 

■  Head-mounted  displays  (HMD). 

■  Standard-size  PC  monitor. 
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o  Auditory  Display  -  Speakers  or  headphones.  They  deliver  environmental  sounds 
and  voice  communications. 

o  Haptic  Display  -  Uses  pressure  or  vibration  to  provide  cues  such  as  physical 
contact  with  environmental  objects  or  people, 
o  Other  Devices  -  To  receive  input  from  additional  equipment  such  as  digital 
systems,  night  vision  goggles,  etc. 

•  Management  System.  This  includes  a  variety  of  capabilities  used  to  set  up  training  scenarios, 
track  unit  and  individual  progress,  control  computer-generated  entities  (people  and  objects), 
and  provide  feedback  (AAR)  to  the  trainees. 


Background  on  Simulators 

It  is  beyond  the  scope  of  this  report  to  provide  a  detailed  review  of  the  use  and 
effectiveness  of  simulators  for  military  training.  However,  the  long  history  of  use  of  simulators 
has  established  both  “conventional  wisdom”  and  empirical  data  about  the  effectiveness  of 
simulators.  Orlansky,  Dahlman,  Hammon,  Metzko,  Taylor,  &  Youngblut  (1994)  provided  a  list 
of  advantages  and  disadvantages  of  the  use  of  simulators.  These  are  cited  as  “conventional 
wisdom,”  which  may  not  always  be  correct  when  applied  to  a  particular  situation. 

Advantages 

•  Trains  many  tasks  as  well  as  the  use  of  the  actual  equipment. 

•  Costs  less  than  actual  equipment  to  procure  and  use. 

•  Provides  training  that  otherwise  requires  use  of  actual  equipment. 

•  Permits  training  for  dangerous  and  potentially  catastrophic  conditions  not  otherwise 
possible  in  actual  equipment. 

•  Reduces  risks,  safety  hazards,  and  wear  and  tear  from  use  and  maintenance  of  actual 
equipment,  and  extends  useful  life  of  equipment  needed  for  combat. 

•  Reduces  impact  that  actual  equipment  has  on  environment,  and  the  noise  and  use  of 
ranges  have  on  communities. 

•  Protects  operational  security  of  tactics  and  mission  rehearsal,  of  electronic  warfare,  and 
sensitive  performance  characteristics  of  new  weapons. 

•  Provides  features  needed  for  instruction  and  feedback,  including  performance  measures. 

•  Saves  on  use  of  fuel,  munitions,  and  support  facilities. 

•  Provides  initial  familiarization,  trains  procedures  (less  so  for  advanced  students),  and 
avoids  risk  to  actual  equipment  by  novices. 

•  Permits  training  not  readily  affordable  with  actual  equipment. 

•  Permits  use  of  actual  or  anticipated  threat  conditions  to  examine  doctrine,  effect  of  the 
combat  environment,  and  potential  responses. 

•  Permits  testing  of  effectiveness  of  plans  and  tactics  prior  to  actual  operations. 

•  Permits  joint  training  of  DoD  forces  and  of  eoalition  forees. 

•  Can  recreate  significant  battles  to  identify  key  events  and  to  train  leaders. 
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Disadvantages 

•  Could  adversely  affect  training  readiness  if  use  of  simulators  reduces  Operational  Tempo 
(OPTEMPO),  as  well  as  confidence  in  use  of  actual  equipment. 

•  If  uninformed  by  data  from  actual  performances  of  weapons  and  service  crews,  virtual 
and  constructive  simulations  can  be  dubiously  valid. 

•  Require  funds  to  procure  and  use. 

•  May  instill  habits  that  are  incorrect  for  use  in  actual  equipment. 

•  Limited  fidelity  of  visual  resolution,  sensors,  aerodynamics,  and  motion  platforms  can 
lead  to  inadequate  or  misleading  training. 

•  Differing  response  characteristics  of  visual  displays  and  of  motion  platforms  often  leads 
to  motion  sickness. 

•  There  is  a  potential  compromise  of  security  if  information  about  sensitive  tactics  and 
equipment  can  be  gained  by  access  to  networks  that  support  interactive  simulators. 

•  Accurate  performance  of  weapons  and  platforms  in  simulations  requires  accurate 
engineering  models  and  data  that  may  not  be  available  or  are  expensive  to  develop. 

Results  of  research  on  the  use  of  simulators  for  military  training 

Orlansky  et  al  (1994),  in  a  review  of  research  on  the  use  of  simulators  for  individual 
flight  and  maintenance  training  in  the  military,  report  that  a  combination  of  real  equipment  and 
simulators  were  always  as  effective  as  use  of  the  actual  equipment  alone,  and  cost  less.  For 
example,  an  hour  of  aircraft  simulator  training  was  equivalent  to  .48  hours  of  actual  aircraft  time, 
while  operating  costs  of  the  simulator  averaged  about  10%  of  those  of  the  actual  aircraft,  and 
total  costs  about  20%  of  actual  aircraft. 

A  review  by  Hayes  et  al  (1992)  (based  on  26  of  247  documents)  concluded  that  the  use  of 
both  a  simulator  and  the  actual  aircraft  for  jet  aircraft  training  consistently  resulted  in  better 
perfonnance  in  the  aircraft  than  did  the  use  of  the  aircraft  for  training  alone.  Caretta  and  Dunlap 
(1998)  reviewed  67  articles,  papers  and  reports  regarding  the  use  of  simulators  for  flying  training 
written  after  the  Hayes  et  al  review.  Of  these,  13  actually  reported  measures  of  transfer  from  the 
simulator  to  the  aircraft.  All  showed  that  at  least  some  of  the  training  received  in  the  simulator 
transferred  to  the  actual  aircraft. 

This  research  summarized  above  is  largely  concerned  with  the  use  of  simulators  for 
individual  training  in  institutions.  However,  as  Orlansky  et  al  point  out,  this  differs  from  one  of 
the  situations  of  interest  to  us,  collective  training  in  units.  First,  the  objective  of  individual 
training  in  institutions  is  to  achieve  standards  of  proficiency  at  the  lowest  possible  cost,  while  the 
objective  of  collective  training  in  units  should  be  to  get  the  best  possible  training  at  little  or  no 
increase  in  cost.  Second,  there  is  very  little  data  to  indicate  how  effective  simulators  and 
simulations  are  for  unit  and  collective  training.  Several  reviewers  (Boldovici,  Bessemer,  and 
Bolton,  2002;  Simpson,  1999)  discuss  the  fact  that  it  is  difficult  and  highly  resource  intensive  to 
do  such  evaluations. 
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Results  of  Research  on  the  Use  of  Immersive  Simulation  for  Training  Dismounted  Leaders, 

Soldiers,  and  Units 

Introduction 

This  section  will  provide  a  review  of  research  on  the  use  of  immersive  simulations  to 
train  Army  personnel  for  tactical  training.  It  will  include  individual  Soldier,  leader,  and 
collective  tasks.  The  criterion  for  including  studies  and  experiments  has  been  fairly  liberal  with 
regard  to  the  definition  of  “immersive”  simulation.  Two  marksmanship  trainers,  one  of  which 
used  videodisc-based  technology,  are  included.  On  the  other  hand,  technology  descriptions  or 
demonstrations  which  obtained  no  structured  feedback  Ifom  trainees  or  subject  matter  experts 
(SMEs)  were  not  included. 

The  Army  began  to  be  interested  in  the  use  of  immersive  simulation  for  dismounted 
Soldier  training  in  the  early  1990’s.  This  was  largely  a  result  of  the  influence  of  Gorman  (1990), 
who  was  an  early  proponent  of  the  use  of  VE  for  dismounted  infantry  (DI)  training.  Partly  as  a 
result  of  his  efforts,  a  conference  was  held  in  Snowbird,  Utah  in  1990  to  discuss  individual 
Soldier  systems  and  the  role  that  an  individual  immersive  simulator,  which  he  termed  an 
individual  portal  (or  I-Port),  would  play  in  their  development  (Goldberg  and  Knerr,  1997).  The 
conference  did  create  an  awareness  of  the  difference  in  simulation  capabilities  between  those 
available  to  support  training  of  Soldiers  fighting  from  vehicles,  and  those  fighting  dismounted. 
Although  consensus  was  achieved  on  the  need  for  an  I-Port,  Operation  Desert  Storm  preempted 
the  initiation  of  a  large-scale  cooperative  research  and  development  program  to  develop  such  a 
simulation  capability.  The  meeting  did,  however,  provide  the  impetus  for  organizations  to  initiate 
Research  and  Development  (R&D)  programs  in  the  area. 

Methodological  Considerations 

The  evaluation  activities  which  followed  can  roughly  be  divided  into  three  categories: 
analyses,  laboratory  experiments,  and  field  experiments. 

In  analyses,  SMEs  use  information  about  the  capabilities  and  characteristics  of  the 
simulation  or  simulator  and  the  tasks  to  be  trained  to  assign  ratings  to  each  task  or  task 
component  (activity)  which  indicate  how  well  they  believe  it  can  be  performed  or  trained  in  the 
simulation.  Rules  can  be  used  to  aggregate  these  ratings  from  task  components  to  higher  level 
tasks.  The  quality  of  the  results  from  an  analytic  approach  is  highly  dependent  on  the  familiarity 
of  the  rater  with  both  the  simulation  and  the  tasks  to  be  trained.  It  also  may  require  making  some 
sophisticated  distinctions  between  being  able  to  perform  a  task  in  a  simulation  and  learning  in  a 
simulation  to  perform  a  task  in  the  real  world.  For  example,  in  one  currently  available  simulator, 
a  Soldier  can  throw  a  simulated  grenade  by  selecting  the  appropriate  type  of  grenade  from  a 
menu,  choosing  direction  and  launch  angle  by  pointing  his  mock  weapon,  choosing  the  amount 
of  force  desired  by  adjusting  a  slider  with  a  pair  of  buttons  on  the  weapon,  and  pulling  the 
trigger.  While  this  bears  little  resemblance  to  the  physical  act  of  throwing  a  real  grenade,  it  does 
provide  practice  in  deciding  which  type  of  grenade  to  use,  and  where  and  when  to  employ  it. 
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In  laboratory  experiments,  trainees  (who  may  or  may  not  be  Soldiers)  perform  or  learn  to 
perform  tasks  under  highly  eontrolled  conditions.  The  tasks  may  be  actual  Soldier  tasks  or 
synthetic  Soldier  tasks  (designed  to  have  the  key  elements  of  Soldier  tasks  but  modified  so  that 
they  can  be  performed  or  learned  by  non-Soldiers).  Laboratory  experiments  compare 
performance  or  learning  under  different  conditions.  Trainee  performance  is  measured 
objectively. 

In  field  experiments,  Soldiers  perform  or  learn  to  perform  Soldier  tasks  under  controlled 
conditions  (although  the  conditions  may  be  less  tightly  controlled  than  under  laboratory 
experiments  because  of  operational  constraints).  There  may  or  may  not  be  multiple  groups  for 
comparison.  Trainee  performance  may  or  may  not  be  measured  objectively. 

Some  experiments  provide  more  convincing  evidence  of  effectiveness  than  others.  There 
are  two  important  aspects  to  the  design  of  an  experiment:  the  training  methods  being  compared, 
and  the  measures  of  training  effectiveness  or  performance  that  are  used.  In  terms  of  the 
comparison  of  training  methods  being  made,  the  best  evidence  of  training  effectiveness  is 
provided  by  those  experiments  which  compare  simulation-based  training  with  an  alternative 
training  approach  and/or  a  control  group  which  receives  no  training.  Experiments  that  compare 
only  pre-training  and  post-training  performance  of  Soldiers  trained  in  a  simulation  provide  less 
convincing  evidence.  In  addition,  some  criteria  used  to  evaluate  training  effectiveness  are  better 
than  others.  The  best  criterion  is  performance  on  similar  tasks  in  a  live  simulation  as  measured 
by  objective  performance  data  or  SME  ratings.  The  next  best  would  be  objeetive  measures  of 
perfonnance  or  SME  ratings  on  the  tasks  as  perfonned  in  the  simulation,  again  as  measured  by 
objective  performance  data  or  SME  ratings.  Self-reports  by  the  trainees  of  what  and  how  much 
they  learned  are  less  informative  than  objective  measures.  Trainees  may  not  be  able  to  assess 
how  much  they  learned  accurately,  and  their  ratings  may  also  be  affected  by  other  factors,  such 
as  the  novelty  of  the  training  situation  or  how  much  they  enjoyed  the  training.  These  self  reports 
can  vary  Ifom  very  general  and  global  to  highly  detailed  and  specific.  The  more  specific  they  are 
the  better. 


Analyses 

The  first  attempts  to  evaluate  the  training  potential  of  immersive  simulations  for  Army 
training  had  to  be  analytical,  since  they  preceded  the  development  of  prototypes  designed  for 
Anny  training.  They  were  instead  based  on  knowledge  of  limited  off-the-shelf  commercial 
systems,  systems  under  development  in  university  laboratories,  and  projections  of  the 
capabilities  of  future  technology.  Levison  and  Pew  (1993)  were  the  first  to  eonduct  an  analysis  of 
simulation  requirements  for  dismounted  Soldier  training.  Their  report  is  of  interest  now  only 
largely  for  historical  reasons,  since  it  was  conducted  at  a  fairly  gross  level  of  detail,  and  VE 
technology  was  primitive  at  the  time,  however,  it  is  interesting  to  note  that  they  saw  benefit  in 
the  use  of  then  current  simulation  technology: 

(Current)  technology  is  considered  adequate  for  most  CPT  (Combat  Proficiency 
Training)  and  MPR  (Mission  Planning  and  Rehearsal)  training  requirements. 

More  sophisticated  visual  display  technology  will  be  required  to  train  in  situations 
that  provide  rapidly-unfolding  activities  in  close  proximity  to  the  Soldier 
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(operations  in  urban  and  other  elose-in  environments),  where  the  resulting  large 
and  rapid  head  movements  plaee  a  higher  demand  on  the  VE  teehnology  needed 
to  supply  the  visual  information  adequate  for  training.  An  advanced  haptic  display 
interface  is  required  for  tasks  that  rely  on  the  sense  of  touch  (such  as  assessing  the 
surface  condition  for  supporting  heavy  equipment),  or  for  situations  where  it  is 
deemed  important  to  have  realistic  tactile  feedback  from  control  and  manipulation 
of  weapons,  equipment,  and  other  physical  objects,  (p.79) 

Levison  and  Pew  (1993)  also  made  a  key  distinction  between  “performance  fidelity”  and 
“training  fidelity”  that  is  still  relevant.  Full  performance  fidelity  is  achieved  when  a  Soldier 
performs  a  task  with  the  same  result  in  the  simulator  as  in  the  real  world,  according  to  any 
objective  measure  of  performance.  Full  training  fidelity  is  achieved  when  training  in  the 
simulator  allows  a  Soldier  to  perform  a  task  in  the  real  world  as  well  or  better  than  if  the  training 
had  been  conducted  in  the  real  world.  Performance  fidelity,  they  said,  would  be  required  for 
mission  planning  and  rehearsal,  and  training  fidelity  would  be  required  for  training. 

Jacobs  et  al  (1994)  followed  up  on  this  work  by  conducting  an  analysis  of  four  Infantry 
and  Special  Forces  Army  Training  and  Evaluation  Plan  Mission  Training  Plans  (ARTEP  MTPs) 
and  drills  (MTP  for  the  Infantry  Rifle  Platoon  and  Squad,  Battle  Drills  for  the  Infantry  Rifle 
Platoon  and  Squad,  MTP  for  the  Special  Forces  Company:  Special  Reconnaissance,  and  MTP 
for  the  Special  Forces  Company:  Direct  Action).  From  these,  they  selected  67  unique  tasks 
which  they  further  broke  down  into  252  individual  Soldier  activities.  They  then  determined  the 
simulation  capabilities  required  in  order  for  each  of  those  activities  to  be  performed  in  a  VE,  and 
compared  this  with  the  then  current  and  projected  state  of  VE  technology.  They  concluded  that 
none  of  the  missions  or  tasks  analyzed  could  be  fully  supported  by  VE  technology  at  that  time, 
but  that  the  great  majority  could  be  partially  supported.  The  list  of  252  individual  Soldier 
activities  has  provided  a  basis  for  subsequent  evaluations  of  what  Soldiers  could  do  in  virtual 
simulations. 

In  retrospect,  it  is  clear  that  VE  technology  has  not  progressed  as  rapidly  as  projected  in 
these  early  analytical  efforts.  Capabilities  in  the  areas  of  speech  recognition,  position  and 
orientation  tracking,  computer  recognition  of  human  gestures,  and  low-cost  high-resolution  wide 
field-of-view  HMDs  has  not  become  available  as  rapidly  as  forecast.  However,  the  emergence 
of  PCs  with  the  capacity  to  drive  the  simulations  and  generate  the  graphics  required  was  not 
projected. 

Perhaps  surprisingly,  the  Navy  was  the  first  service  to  produce  a  prototype  of  a  virtual 
individual  combatant  simulator.  The  Team  Tactical  Engagement  Simulator  (TTES),  a  graphic  of 
which  is  shown  in  Figure  1,  consisted  of  a  8’  X  10’  rear  projection  display,  demilitarized  rifle 
and  tracker,  head  tracker,  computer  graphics  generator,  and  system  software.  Trainees  moved 
through  the  virtual  world  through  the  use  of  a  foot  pedal;  pressure  on  the  front  of  the  pedal 
moved  the  trainee  forward  (in  the  direction  of  gaze),  while  pressure  on  the  back  of  the  pedal 
moved  the  trainee  in  the  opposite  direction.  TTES  was  expected  to  serve  multiple  applications: 
individual  marksmanship  trainer,  individual  and  unit  tactical  trainer,  and  decision  making  trainer. 
TTES  was  first  evaluated  in  1994  (Lind  and  Adams,  1994)  by  21  military  and  civilian  SMEs 
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who  rated  the  importance,  current  eapability,  and  potential  capability  of  TTES  for  each  of  those 
three  applications.  Each  application  was  defined  by  multiple  events  or  activities  which  were 


Figure  1 .  A  graphic  depiction  of  the  TTES  (From  Lind  and  Adams,  1994). 

rated  separately.  Prior  to  providing  their  ratings,  SMEs  received  a  briefing  describing  TTES  and 
projected  future  enhancements,  used  the  simulator  for  approximately  25  minutes,  and  had  the 
opportunity  to  observe  others  using  it.  Major  results  are  summarized  in  Table  2.  Ratings  were 
aggregated  and  converted  so  that  the  numbers  shown  in  the  table  are  expressed  as  a  percentage  of 
the  highest  possible  rating.  Lind  and  Adams  recommended  as  a  result  that  emphasis  be  placed  on 
developing  the  functions  that  had  the  perceived  highest  importance  and  potential  capability 
ratings.  Marksmanship  and  Discretionary  Decisions.  However,  it  is  also  clear  that  the  SMEs 
considered  the  current  version  of  TTES  to  be  well  short  of  its  potential. 


Table  2. 

1994  SME  Ratings  of  TTES  Functions 


Function 

Importance 

Potential 

Capability 

Current 

Capability 

(Tactical)  Training  Situations 

76.4 

76.4 

45.0 

Marksmanship 

85.0 

80.0 

46.3 

Discretionary  Decisions 

83.5 

85.5 

41.3 

Note:  The  numbers  in  the  cells  represent  the  percentage  of  the  maximum  possible  rating  assigned 
to  the  events  in  that  category. 


12 


Lind  (1995)  conducted  a  similar  evaluation  one  year  later  in  which  28  SMEs  participated. 
Proeedures  were  generally  the  same,  although  not  all  of  the  SMEs  participated  in  the  TTES 
exercises  (some  merely  observed  the  participation  of  others),  and  those  who  participated  did  so 
as  two-man  teams  using  two  networked  TTES,  rather  than  as  individuals.  The  questionnaire  was 
changed  slightly.  The  list  of  functions  was  expanded  to  include  Mission  Preview  and  Mission 
Rehearsal,  and  “Importance”  and  “Potential  Capability”  were  replaeed  by  “Potential 
Usefulness.”  The  results  are  shown  in  Table  3.  Based  on  the  high  Potential  Usefulness  ratings 
(93.9%  and  higher),  Lind  concluded  that  simulators  such  as  TTES  could  produce  “phenomenal 
savings  in  time,  ammunition,  and  other  costs  related  to  training,  while  providing  the  eapability  to 
train  to  higher  standards  and  to  ehallenge  trainees  in  ways  not  currently  possible”  (p.3).  It  is 
perhaps  more  significant  to  note  that  the  Current  Capability  ratings  for  the  (Tactical)  Training 
Situations,  Marksmanship,  and  Discretionary  Decisions  functions  increased  substantially  from 

1994  to  1995,  presumably  a  result  of  improvements  in  the  TTES. 

Table  3. 

1995  SME  Ratings  of  TTES  Functions 


Function 

Potential  Usefulness 

Current  Capability 

(Tactical)  Training  Situations 

96.4 

69.5 

Marksmanship 

95.0 

71.1 

Discretionary  Decisions 

95.2 

70.0 

Mission  Preview  (walkthrough) 

94.8 

72.6 

Mission  Rehearsal 

93.9 

73.0 

Note:  The  numbers  in  the  eells  represent  t 

le  percentage  of  the  maximum  possible  rating 

assigned  to  the  events  in  that  category. 

These  two  TTES  evaluations  probably  best  serve  to  document  the  fact  that  SMEs 
perceived  TTES  to  have  the  potential  to  provide  effective  training  in  a  variety  of  areas.  The 
TTES  implementation,  however,  was  perceived  to  be  short  of  that  potential.  Most  importantly, 
no  actual  training  was  conducted  or  evaluated. 

Experimental  Efforts 


Marksmanship  trainers 

Eisley,  Ragman,  Ashworth,  and  Viner  (1990)  conducted  an  evaluation  of  the  Squad 
Engagement  Training  System  (SETS)  as  both  an  individual  marksmanship  trainer  and  a  squad 
tactical  trainer.  While  SETS  does  not  use  computer-generated  graphies,  it  eould  be  considered  to 
be  immersive.  It  uses  computer-controlled,  videodisc-based  visual  imagery  projected  on  a  wide 
screen,  and  laser-instrumented  individual  weapons  to  train  both  individual  marksmanship  and  to 
a  limited  extent,  squad  tacties.  Conceptually,  it  seems  very  similar  to  the  current  Engagement 
Skills  Trainer  (EST  2000).  This  evaluation  provides  sound  evidenee  that  simulators  can  provide 
effeetive  tactieal  training  for  small  dismounted  units. 

Thirty-six  squads  of  National  Guard  Soldiers  (18  Infantry  and  18  support)  were  randomly 
assigned  to  either  the  experimental  or  the  control  group.  Experimental  group  squads  received 
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two  hours  of  SETS  training  that  included  both  simulated  firing  for  record  and  two  tactical 
defensive  scenarios.  They  then  zeroed  their  weapons  on  a  live-fire  range,  conducted  two  tactical 
live-fire  exercises,  and  then  fired  for  record  on  a  live-fire  range.  Control  group  squads  had  no 
SETS  training.  They  only  zeroed  their  weapons  on  a  live-fire  range,  conducted  two  tactical  live- 
fire  exercises,  and  then  fired  for  record  on  a  live-fire  range.  Squad  Leader  performance,  as  rated 
on  27  tasks  extracted  from  ARTEP  standards,  improved  from  the  first  to  the  second  SETS 
exercise,  and  was  higher  on  the  live-fire  exercises  for  the  experimental  group  than  the  control 
group.  This  difference  was  particularly  apparent  for  the  leaders  of  the  support  squads.  The  fire 
distribution  of  experimental  group  squads  on  the  live  fire  test  was  also  superior  to  that  of  the 
control  group  squads.  While  the  qualification  scores  of  the  Soldiers  in  the  experimental  group 
were  slightly  (but  significantly)  higher  than  those  of  Soldiers  in  the  control  group,  the 
relationship  between  live  fire  and  SETS  qualification  scores  was  not  sufficiently  strong  that 
SETS  could  be  recommended  as  a  substitute  for  live  fire  qualification.  The  authors  concluded 
that  SETS  had  the  potential  to  support  home-station  training  of  basic  defensive  tactical  skills. 

This  experiment  showed  conclusively  that  SETS  provided  a  training  benefit  for  leaders, 
Soldiers,  and  units  that  transferred  to  the  real  world.  However,  this  benefit  was  achieved  through 
two  hours  of  additional  training.  There  is  no  indication  of  how  this  would  compare  with  two 
hours  of  live  training.  The  authors  also  stressed  the  instructor  dependence  of  SETS-based 
training.  It  is  up  to  the  instructors  to  establish  the  training  context,  determine  how  to  evaluate 
leader  performance,  and  evaluate  it. 

In  a  related  evaluation  with  more  current  technology,  Miller  and  Zeisset  (2002) 
conducted  a  Post-Fielding  Training  Effectiveness  Evaluation  of  the  EST  2000.  The  EST  2000  is 
a  small  arms  simulator  design  to  augment  and  substitute  for  individual  marksmanship  training 
and  squad  collective  training.  It  provides  simulated  weaponry  that  simulates  the  physical  and 
operational  characteristics  of  several  weapons.  Unlike  the  Eisley  et  al  (1990)  experiment,  whieh 
used  the  simulator  as  a  means  of  providing  additional  training,  Miller  and  Zeisset  compared  a 
control  group  which  trained  using  live  fire  proeedures  with  an  experimental  group  that 
substituted  EST  2000  training  for  a  portion  of  the  live  fire.  The  extent  of  the  training,  either  in 
tenns  of  the  number  of  rounds  fired  or  time  required,  was  not  described.  The  largest  number  of 
Soldiers  trained  on  the  M16A2  384  using  live  fire  and  76  using  the  EST.  There  were  no 
significant  differences  in  the  qualification  scores  between  the  EST  and  the  live  fire  groups,  either 
in  tenns  of  mean  scores  or  in  the  proportion  qualifying  as  Expert,  Sharpshooter,  or  Marksman. 
The  results  for  the  M240B  machine  gun  were  similar,  in  that  no  significant  differences  were 
found,  although  the  sample  sizes  were  so  small  (22  Soldiers  in  each  group)  that  little  confidence 
can  be  placed  in  this  conclusion. 

The  Dismounted  Warrior  Network  Experiments 

Involvement  of  Army  Soldiers  in  what  we  would  today  think  of  as  immersive  virtual 
simulation  began  with  the  Dismounted  Warrior  Network  (DWN)  program  in  1997.  DWN  was  a 
U.S.  Army  Simulation,  Training  and  Instrumentation  Command  (STRICOM)'  program  to 
develop  a  reliable,  low-cost,  easy-to-use  capability  to  insert  dismounted  Soldiers  into  VE.  The 


'  What  was  then  the  R&D  component  of  STRICOM  is  now  the  U.S.  Army  Research,  Development,  and  Engineering 
Command  Simulation  and  Training  Technology  Center. 
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DWN  program  (and  its  successor,  DWN  Enhancements  for  Restricted  Terrain  (ERT))  did  not 
evaluate  training  effeetiveness,  but  did  obtain  data,  through  automated  data  eapture.  Soldier 
questionnaires,  and  observation,  about  task  performance  in  the  virtual  simulators  used  in  the 
program. 

A  series  of  experiments  was  conducted  during  1997  to  investigate  different  interfaces  for 
inserting  dismounted  Soldiers  into  virtual  simulations.  Four  Virtual  Individual  Combatant 
Simulators  (VICS)  were  selected  for  evaluation  based  on  three  criteria:  a  desire  to  have  a  diverse 
mixture  of  characteristics  to  examine;  a  cost/benefit  assessment  of  system  characteristics;  and 
system  availability.  The  VICS  were  designated  Alpha,  Bravo,  Charlie,  and  Foxtrot. 


Figure  2.  A  Soldier  in  VIC  Alpha. 

VIC  Alpha  used  full  body  tracking  with  multiple  fixed-position  video  cameras  and 
reflective  markers  on  the  head,  body,  and  weapon  to  track  posture,  orientation  and  weapon 
aiming.  Locomotion  used  a  “human  joystick”  approach,  with  movement  direction  determined  by 
body  orientation,  and  speed  of  movement  determined  by  distance  from  a  marked  center  point  on 
the  floor;  the  further  the  user  was  from  the  center  point,  the  faster  the  movement.  Return  toward 
the  center  point  was  required  to  slow  down  or  stop.  A  low  resolution  (420  x  230  pixels)  wireless 
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HMD  was  used  to  provide  a  view  of  the  virtual  world.  A  photograph  of  a  Soldier  using  VIC 
Alpha  is  shown  in  Figure  2.  The  white  spheres  are  the  reflective  markers  for  the  tracking  system. 

VIC  Bravo  projected  visual  images  on  four  rear-projection  screens  that  surrounded  the 
user.  Locomotion  was  provided  by  the  Omni-Directional  Treadmill  (ODT),  which  used 
interlocked  belts  oriented  at  right  angles  in  conjunction  with  a  centering  harness  to  permit  the 
user  to  walk  in  any  direction  while  remaining  in  the  same  position.  A  Soldier  in  VIC  Bravo  is 
shown  in  Figure  3. 


Figure  3.  A  Soldier  in  VIC  Bravo. 

There  were  two  versions  of  VIC  Charlie,  one  used  during  the  Engineering  Experiments 
and  the  other  used  during  the  User  Experiments.  While  they  had  different  simulation  software, 
the  interface  for  both  was  a  computer  monitor  and  joystick. 

VIC  Foxtrot  was  the  then-current  version  of  the  Navy  TTES,  described  earlier  and  shown 
in  Figure  1 .  It  used  a  foot  pedal  for  control  of  movement,  a  single  rear  projection  screen,  and  a 
tracking  system  to  detennine  head  and  weapon  position  and  orientation. 
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The  DWN  evaluations  were  conducted  in  two  phases.  Eight  Soldiers  participated  in  each. 
The  first  was  a  set  of  Engineering  Experiments,  which  focused  on  how  well  individual  Soldier 
tasks  (detect  targets,  engage  targets,  move  through  a  building,  etc)  could  be  performed  in  the 
simulators.  The  Engineering  Experiments  made  extensive  use  of  objective  measures  of  Soldier 
performance.  After  the  Engineering  Experiments  were  completed,  the  VICS,  a  DI  Semi- 
Automated  Forces  (DISAF)  station,  an  Exercise  Support  Station,  and  an  AAR  station  were  tied 
into  a  DIS  network  and  installed  at  the  Soldier  Battle  Lab  (SBL)  ,  Fort  Benning.  A  set  of  User 
Experiments  was  then  conducted  which  focused  on  the  performance  of  team  and  squad  taetical 
tasks.  The  User  Experiments  relied  heavily  on  the  use  of  questionnaires  to  obtain  feedbaek  from 
Soldiers  regarding  how  well  they  could  perform  tasks  in  the  simulators.  Soldiers  rated  each  VIC 
on  the  extent  to  which  they  could  perform  selected  tasks,  and  the  difficulty  of  performing  those 
tasks.  Tasks  covered  movement,  orientation,  visual  recognition,  and  weapon  engagement.  The 
detailed  results  are  presented  in  Lockheed  Martin  (1997)  and  Pleban,  Dyer,  Salter,  &  Brown 
(1998). 


The  DWN  experiments  were  successful  in  showing  that  Soldiers  could  perform  basic 
Infantry  tasks  in  the  simulators.  All  VICS  were  somewhat  effective  for  engaging  targets, 
simulating  locomotion,  and  identifying  people  and  objects.  However,  the  experiments  also 
revealed  many  limitations  of  the  VICS.  Soldier  performance  on  visual  tasks  was  closely  related 
to  the  resolution  of  the  visual  display:  the  higher  the  resolution,  the  better  the  performanee. 
Weapon  use  was  difficult  to  implement  with  an  HMD.  It  required  presenting  a  graphic  image  of 
the  weapon  in  the  HMD  in  a  position  that  corresponded  with  the  actual  position  of  the  weapon 
with  little  margin  for  error.  In  addition,  the  bulk  of  the  HMD  prevented  the  Soldier  from 
bringing  the  weapon  into  its  proper  aiming  position.  Some  of  the  interfaces  were  difficult  to 
master.  DISAF  did  not  work  in  buildings.  System  reliability  was  a  major  factor  in  determining 
Soldier  preferences.  Soldiers  preferred  VIC  Bravo  over  VIC  Alpha,  but  otherwise  there  were  no 
differences. 

A  follow-on  project  to  DWN,  entitled  DWN  ERT  focused  on  Military  Operations  in 
Urban  Terrain  (MOUT).  Some  VICS  were  modified  and  new  ones  developed  based  on  lessons 
learned  in  the  DWN  experiments.  New  locomotion  methods  were  introduced,  improved  visual 
systems  were  incorporated,  and  new  aiming  techniques  were  implemented.  In  addition,  DISAF 
was  modified  to  support  operations  inside  buildings.  Engineering  experiments,  live  tests  at  the 
Fort  Benning  MeKenna  MOUT  site,  and  user  experiments  were  condueted  in  July  1998  with 
these  modified  systems.  The  goal  of  this  round  of  experiments  was  to  investigate  how  well  a  fire 
team  of  VICS  and  DISAF  could  support  MOUT  tasks  at  the  individual  Soldier,  fire  team,  and 
squad,  levels. 

The  four  VICS  ineluded  an  improved  version  of  VIC  Alpha  from  the  1997  experiments, 
and  three  VICs  based  on  the  Soldier  Visualization  Station  (SVS™).  Improvements  to  VIC 
Alpha  included  changing  locomotion  control  from  the  “human  joystick”  to  a  weapon-mounted 
joystick,  and  upgrading  the  HMD.  All  SVS^'^-based  VICs  used  two  PCs  for  calculations  and 
image  generation,  a  demilitarized  weapon,  and  a  tracking  system  to  determine  head  position  and 
weapon  position  and  orientation.  VIC  Delta  was  the  basie  SVS™  used  with  a  10’  x  8’  rear 
projection  screen  and  a  rifle-mounted  joystick  for  movement.  VIC  Echo  was  the  basic  SVS™ 


^  This  is  the  current  name  for  the  facility.  There  have  been  several  name  changes  since  its  founding. 
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used  with  a  front  projection  dome  display  which  provided  a  field  of  view  approximately  150 
degrees  wide  by  40  degrees  high.  As  in  VIC  Delta,  movement  was  controlled  by  a  rifle-mounted 
joystick.  VIC  Golf  used  the  SVS™  software  with  an  HMD  and  the  ODT  from  the  1997 
experiments. 

Eight  Soldiers  participated  in  tests  of  the  systems  over  a  two  week  period.  The  detailed 
results  are  documented  in  the  DWN  ERT  Final  Report  (Lockheed  Martin  Corporation,  1998)  and 
Salter,  Eakin,  &  Knerr  (1999).  The  results  are  to  some  extent  tied  to  the  specific  (and  now 
outdated)  equipment  employed,  but  they  also  serve  to  illustrate  some  trends  that  continue  to  the 
present.  First,  locomotion  in  the  virtual  world  was  difficult.  It  took  the  Soldiers  approximately 
four  times  as  long  to  complete  a  route  through  a  virtual  building  than  it  did  to  complete  the  same 
route  in  the  aetual  building  at  the  McKenna  MOUT  site.  Inability  to  track  weapon  positions 
accurately  led  to  the  weapon  aiming  portion  of  the  test  being  curtailed.  Even  at  the  shortest 
range  (25  meters),  the  best  VICs  hit  only  67%  of  the  targets.  In  addition,  the  accuracy  of  the 
trackers  decreased  with  the  distance  of  the  weapon  from  the  tracker  (which  was  mounted 
overhead).  This  resulted  in  the  prone  position  being  the  least  accurate  firing  position. 

In  both  the  engineering  and  user  experiments,  the  Soldiers  rated  the  VICS  on  the 
dimensions  of  similarity  to  the  real  world,  and  speed,  quality,  and  difficulty  of  performance.  In 
the  engineering  experiments,  the  activities  rated  were  Detect  Targets,  Engage  Targets,  Move 
Inside  Buildings  and  Move  Outside  Buildings.  In  the  user  experiments  the  activities  rated  were 
Tactical  Movement,  Room  Clearing,  React  to  Contact/Engage,  and  Communicate.  When  there 
were  significant  differences  among  the  VICs,  it  generally  indicated  that  VIC  Golf,  with  the 
combination  of  an  HMD  and  the  ODT,  performed  less  well  than  the  others.  VIC  Golf  also 
produced  more  symptoms  of  simulator  sickness.  (See  separate  section  on  simulator  sickness). 

Salter,  Eakin,  and  Knerr  (1999)  identified  four  major  areas  for  improvement.  The  first 
was  position  and  orientation  tracking.  Improved  tracking  was  necessary  to  improve  weapons 
accuracy.  The  second  was  the  means  of  simulating  locomotion.  While  walking  in  the  real  world 
is  natural  and  is  usually  accomplished  without  much  conscious  effort,  “walking”  in  the  virtual 
world  was  not.  Within  the  amount  of  time  available,  Soldiers  did  not  acquire  full  proficiency  in 
the  simulators  and  consequently  moved  more  slowly  in  the  virtual  world  than  in  the  real  world. 
They  also  frequently  collided  with  walls.  Locomotion  required  conscious  effort  that  may  have 
impaired  the  perfonnance  of  other  tasks  or  interfered  with  training.  The  third  was  providing  a 
means  of  non-verbal  communication,  such  as  gestures  and  facial  expressions.  The  fourth  was 
wide-angle  visual  displays. 

The  Virtual  Environment  Science  and  Technology  Objective  (STO) 

During  the  period  1 999  -  2005  there  were  a  series  of  related  technology  development  and 
transition  activities  which  shared  a  common  assessment  methodology  and  used  nearly  identical 
measures  of  task  performance  and  training  effectiveness  in  VE.  In  the  following  sections,  the 
assessment  activities  will  be  described,  and  then  the  collective  results  twill  be  described  and 
discussed. 
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Following  the  completion  of  the  DWN  experiments,  a  four-year  (FY  1999-2002)  STO 
effort  was  initiated  to  develop  a  demonstration  virtual  dismounted  leader  trainer  at  the  fire  team, 
squad,  and  platoon  level.  Five  Army  organizations^  collaborated  on  the  research  and 
development.  Each  had  a  particular  area  of  interest,  but  all  worked  together  to  explore  concepts 
and  systems  and  to  recommend  directions  for  further  work  on  training,  concept  development, 
and  mission  rehearsal. 

One  of  the  emphases  during  the  first  year  of  the  STO  (FY  1999)  was  on  identifying  high- 
payoff  small  unit  and  leader  training  applications.  Pleban,  Eakin,  and  Salter  (2000)  took  the  task 
lists  from  analytic  efforts  (including  Jacobs  et  al,  1994),  and  eliminated  those  tasks  that  could  not 
be  supported  by  the  technology  as  represented  in  the  then  currently  available  SVS™;  (VIC  Delta 
in  the  DWN  ERT  experiments).  Tasks  were:  move  through  built-up  areas,  reconnoiter  area, 
react  to  contact,  assault,  and  clear  a  building.  They  then  refined  their  findings  by  having  six 
Soldiers  carry  out  1 5  tactical  scenarios  in  the  simulators  as  three-man  teams,  and  then  additional 
scenarios  as  a  six-man  team.  After  completing  a  set  of  three  similar  scenarios,  they  rated  their 
ability  to  perform  each  of  the  component  activities  of  that  task  on  a  four-point  scale  (Very  Poor, 
Poor,  Good,  or  Very  Good).  Ratings  were  then  aggregated  to  produce  an  overall  rating  for  each 
task.  The  activities  and  the  rating  scheme  developed  by  Pleban  Eakin,  and  Salter  (2000)  formed 
the  basis  for  Soldier  ratings  of  their  ability  to  perform  tasks  in  virtual  simulations  in  a  number  of 
subsequent  investigations  performed  during  the  conduct  of  the  STO  and  even  subsequent  efforts. 

Each  year  of  the  STO  a  culminating  event  (CE)  was  conducted.  The  CEs  were 
comprehensive  demonstrations  and  assessments  conducted  with  Soldiers,  using  as  much  of  the 
developed  technology  as  was  feasible  in  a  realistic  training  exercise.  Three  of  the  four  CEs 
(1999,  2001,  and  2002)  were  conducted  at  the  SBL  and  will  be  discussed  here. 

The  1999  CE  consisted  of  squad-level  exercises  which  were  conducted  on  three 
consecutive  days,  with  different  squads  involved  on  each  of  those  days.  All  Soldiers  participated 
using  what  was  then  called  the  Squad  Synthetic  Environment  (SSE),  a  network  of  individual 
simulators,  each  of  which  consisted  of  a  SVS  and  a  virtual  radio.  The  SVS  is  a  PC  based, 
DIS  compatible  DI  simulator  developed  by  Reality  By  Design  (RBD),  and  was  essentially  VIC 
Delta. 


Squads  generally  consisted  of  a  Squad  Leader  and  two  fire  teams.  Each  fire  team  was 
composed  of  three  Soldiers  and  a  fire  team  leader.  The  focus  of  the  evaluation  was  as  the  squad 
leader  as  the  primary  trainee.  Five  exercises  were  conducted  for  each  squad,  although  not  all 
squad  members  were  involved  in  each.  The  first  and  fifth  exercises  were  assessment  exercises 
and  the  entire  squad  participated  in  both.  The  second,  third,  and  fourth  exercises  were  conducted 
with  different  mixes  of  virtual  Soldiers  and  DISAF.  Each  squad  leader  conducted  exercises  with 
an  all-virtual  squad,  an  all-DISAF  squad,  and  a  Half-SAF  squad  (one  virtual  fire  team  and  one 
DISAF  fire  team).  AARs  and,  to  a  limited  extent,  coaching  were  used  improve  learning. 


^  The  organizations  were  the  U.S.  Army  Research  Institute  Simulator  Systems  and  Infantry  Forces  Research  Units, 
the  U.S.  Army  Simulation  Training  and  Instrumentation  Command,  and  the  U.S.  Army  Research  Laboratory  Human 
Research  and  Engineering  and  Computational  and  Information  Sciences  Directorates. 
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At  the  completion  of  their  exercises,  all  Soldiers  who  were  not  squad  or  fire  team  leaders 
rated  their  ability  to  perform  48  tasks  (tfom  Pleban  Eakin,  and  Salter,  2000)  in  the  simulators  as 
very  poor,  poor,  good,  or  very  good.  Squad  and  fire  team  leaders  instead  completed  a 
questionnaire  which  asked  them  to  rate  their  improvement  in  1 1  areas  as  a  result  of  their  training. 
The  areas  were;  Plan  a  Tactical  Operation,  Coordinate  Activities  with  your  Chain  of  Command, 
Assess  the  Tactical  Situation,  Control  Squad  or  Fire  Team  Movement  during  the  Assault, 
Communicate  with  Members  of  your  Team  or  Squad,  Clear  a  Building,  Clear  a  Room,  Control 
Squad  or  Fire  Team  while  not  in  Contact  with  the  Enemy,  Control  your  Squad  or  Fire  Team, 
React  to  Contact  Battle  Drill,  and  Locate  Known  or  Suspected  Enemy  Positions.  These  same 
questionnaires,  with  minor  variations  in  the  Soldiers  tasks,  would  be  used  repeatedly  in 
subsequent  assessments,  and  provide  a  basis  for  year-to-year  comparisons. 

By  the  third  year  of  the  STO  (2000-2001),  the  focus  was  broadened  to  include  a  heavy 
emphasis  on  the  human  interface  aspects  of  the  VE  systems  for  training,  including  the  use  of 
automatic  speech  recognition  to  control  DISAF,  development  of  an  AAR  system,  and 
incorporation  of  a  broad  range  of  DISAF  behaviors.  At  the  same  time,  the  technologies  were 
becoming  more  mature  and  stable,  and  interoperability  issues  were  being  addressed  with 
increasing  success.  Training  continued  to  be  a  principal  area  of  investigation. 

The  FY  01  CE  was  held  in  September  2001  at  the  SBL  (Knerr  et  al  2002).  The  key 
technologies  included:  a  Dismounted  Infantry  Virtual  AAR  System;  new  behaviors  and 
improved  operator  control  for  DISAF;  fire  team  leader  control  of  DISAF  through  Voice 
Recognition  and  Synthesis;  enhancements  to  the  SVS™;  an  improved  ODT;  and  a  dynamic 
terrain  server. 

Infantry  Soldiers  participated  in  scenarios  in  the  simulators,  with  one  group  of  six 
Soldiers  participating  for  one  day,  and  two  groups  of  six  soldiers  participating  for  two  days  each. 
The  network  configuration  for  the  CE  is  shown  in  Figure  4.  The  following  items  were  coimected 
to  the  network: 

•  Five  SVS™  individual  Soldier  simulators.  These  were  used  by  the  squad  leader,  the 
two  fire  team  leaders,  and  two  of  the  three  Fire  Team  A  members.  The  simulators  were 
identical,  except  for  additional  equipment  in  the  Fire  Team  B  leader’s  area  for  the  voice 
recognition  system.  All  SVSs  were  equipped  with  ASTi™  radio  headsets,  which 
permitted  verbal  communication  on  up  to  two  channels,  depending  on  the  duty  position. 
The  squad  leader  could  talk  to  the  platoon  leader  (a  role  player)  and  his  fire  team  leaders. 
Each  fire  team  leader  could  talk  to  the  squad  leader  and  his  subordinates.  Fire  team 
members  could  talk  among  themselves  and  with  their  fire  team  leader. 

•  One  ODT  station.  The  third  A  fire  team  member  used  this  simulator.  Like  the  SVSs, 
this  station  was  equipped  with  an  ASTi™  radio  headset. 

•  One  Voice  Recognition  PC. 

•  One  AAR  System  PC. 

•  One  Dynamic  Terrain  Server. 

•  One  BattleMaster/DISAF  Operator  Station.  The  DISAF  Operator  and  the  Exercise 
Controller  used  this  station. 

•  One  Desktop  SVS™  used  by  a  role  player. 
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Figure  4.  2001  CE  System  Configuration,  Soldier  Battle  Lab,  Fort  Benning.  The  2002  System 
Configuration  was  similar,  except  the  DlSim  with  ODT  was  replaced  with  an  additional  SVS™. 


A  general  situation  and  eight  different  scenarios  were  developed.  Each  of  the  scenarios 
was  designed  to  be  about  20  minutes  in  duration.  They  focused  specifically  on  forcing  the 
Soldiers  to  use  the  various  new  devices  and  capabilities  in  the  VE.  Each  scenario  provided  a 
different  mission  so  the  capabilities  would  be  used  under  differing  circumstances.  Scenarios 
were  also  designed  to  actively  involve  as  many  squad  members  as  possible.  Seven  of  the  eight 
scenarios  took  place  in  a  virtual  representation  of  the  Shugart-Gordon  MOUT  facility  at  Ft.  Polk, 
LA.  The  eighth  scenario  took  place  in  a  high-rise  office  building  described  as  being  in  an 
adjacent  town.  The  scenarios  covered  a  variety  of  wartime  and  SASO.  The  descriptive  scenario 
titles  were:  Support  Operations  Checkpoint,  Hostage  Rescue,  Support  by  Fire,  Assault  and  Clear 
a  Building,  Roving  Patrol,  Air  Assault  and  Clear  a  Building,  Crowd  Control,  and  Downed 
Helicopter. 


In  the  final  year  of  the  STO  (FY  2002),  there  was  a  concerted  effort  to  document  the 
training  potential  of  the  systems,  and  to  prepare  recommendations  for  the  road  ahead.  The  FY  02 
CE  was  held  in  August  2002  at  the  SBL  (Knerr  et  al,  2003).  Both  the  hardware  and  software 
configuration  and  the  scenarios  were  similar  to  those  of  2001 .  Also,  three  squads  participated  for 
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two  days  each.  Two  different  versions  of  the  Hostage  Rescue  and  Deliberate  Attack  scenarios 
were  designed  to  be  comparable  in  terms  of  mission  type  and  difficulty,  although  the  starting 
points  and  objectives  were  different.  By  using  these  parallel  versions,  researchers  were  able  to 
identify  improvements  in  performance  or  effects  of  simulation  fatigue  across  time.  In  addition  to 
the  Leader  ratings  of  changes  in  their  performance  that  had  been  obtained  previously,  SMEs 
rated  squad  leader  performance  on  14  different  factors  during  each  scenario. 

The  Virtual  -Integrated  MOUT  Training  System  (V-IMTS)  Assessment 

V-IMTS  was  a  short-term  project  to  speed  the  transition  of  the  virtual  simulation 
technology  developed  under  the  STO  that  specifically  considered  the  integration  of  live  and 
virtual  training.  The  objective  of  the  assessment  was  to  obtain  information  about  the  performance 
of  Soldier  tasks  and  training  effectiveness  in  the  V-IMTS  configuration. 

The  Cassidy  Combined  Arms  Collective  Training  Center  at  Ft.  Campbell,  KY  was  the 
initial  transition  site  for  V-IMTS  and  the  site  for  the  assessment  (Knerr  and  Lampton,  2005).  It 
consists  of  a  28-building  complex  of  one-  to  four-story  buildings  representing  a  small  town.  As 
part  of  the  V-IMTS  program,  a  deployable  shelter  approximately  40  feet  square  was  placed  next 
to  the  Cassidy  control  center.  Three  Immersive  SVS'*^  simulators  (SVSl)  and  six  Desktop 
SVS™  simulators  (SVSD)  were  installed  inside.  The  squad  leader  and  two  fire  team  leaders 
used  the  SVSI.  The  remaining  Soldiers  used  the  SVSDs  located  immediately  behind  their  fire 
team  leader.  DIVAARS,  DISAF,  and  SVSD  stations  for  human  opposing  forces  were  located  in 
the  control  room. 

The  assessment  took  place  in  September  2004.  Twenty-seven  Soldiers  from  three  squads 
participated.  The  squads  were  actual  units,  not  groups  formed  specifically  for  this  event.  ViTiile 
68%  of  the  squad  members  had  served  a  tour  in  Iraq  or  Afghanistan,  all  three  squad  leaders,  and 
three  of  the  six  team  leaders,  had  served  in  their  current  duty  position  for  one  month  or  less. 

Their  unit  personnel,  assisted  by  technical  personnel,  controlled  the  exercises  and  conducted  the 
AARs.  The  scenarios  consisted  of  multiple  variants  of  two  missions:  Search  and  Cordon  a 
Building  and  Attack/Assault  a  Building.  The  variants  differed  in  the  buildings  involved  and  the 
positions  and  actions  of  the  enemy.  Each  exercise  consisted  of  the  delivery  of  the  operations 
order  to  the  squad  leader,  a  planning  period,  mission  execution,  and  an  AAR  (usually  conducted 
by  the  Platoon  Leader). 
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Figure  5.  V-IMTS  physical  configuration. 

Each  squad  completed  two  live  exercises  separated  by  two,  three,  or  six  virtual  exercises. 
Squad  1  participated  in  six  virtual  exercises  lasting  a  total  of  96  minutes  (mean  =  16.0  minutes), 
squad  2  participated  in  four  virtual  exercises  lasting  a  total  of  77  minutes  (mean  =  19.2  minutes), 
and  squad  3  participated  in  three  virtual  exercises  lasting  a  total  of  44  minutes  (mean  =  14.7 
minutes).  Overall  mean  duration  of  the  virtual  exercises  was  16.7  minutes. 

Situational  Awareness  in  Virtual  Environments  (SAVE) 

The  SAVE  assessment  was  conducted  at  the  SBL  in  2005.  The  purpose  of  the 
assessment  was  to  try  out  tools  for  collection  data  on  Squad  Leader  Situation  Awareness. 
However,  since  it  involved  full  squads  conducting  tactical  exercises  in  immersive  simulators  (the 
SVSI),  Soldier  rating  of  simulator  capabilities  and  Soldier  and  leader  ratings  of  training 
effectiveness  were  obtained.  Each  of  three  squads  participated  in  three  building  clearing 
exercises  and  six  urban  operations  exercises  (three  security  patrol  and  three  search  for  arms 
cache)  over  a  two-day  period.  Average  scenario  duration  was  about  30  minutes. 

Results 


Table  4  summarizes  the  results  of  the  immersive  simulator  eapability  ratings  for  the 
assessments/exercises  described  above  conducted  during  the  period  2002  -  2005.  The  eomplete 
breakout  of  the  ratings  for  each  assessment  is  shown  in  Appendix  X.  Fifty-three  Soldiers  and 
leaders  participated  in  those  exercises  using  immersive  simulators.  While  simulation  technology 
in  general  and  the  SVS^”^  in  particular  were  evolving,  the  capabilities  were  fairly  static  during 
the  2002-2005  period,  and  for  this  reason  those  ratings  have  been  combined  to  produce  a 
summary  rating.  Activities  are  ordered  from  best  (Very  Good,  3.0)  to  worst  (Very  Poor,  0.0). 
Thirty  of  the  fifty-four  activities  were  rated  Good  or  better  (2.00  and  above);  eighteen  were  rated 
between  the  Good/Poor  midpoint  (1.50)  and  Good  (2.00);  and  six  were  rated  Poor  (1.00-1.49). 
Activities  which  were  rated  highly  included  outdoor  movement,  identifieation  of  types  of  people 
(civilians,  non-combatants  within  a  room,  enemy  soldiers),  identification  of  tactically  significant 
areas  (seetors  of  observation  and  responsibility),  and  individual  weapons  use  (but  not  grenades). 
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Table  4. 

Combined  Ratings  of  Simulator  Capability  (2002  -  2005) 


Move  through  open  areas  as  a  widely  separated  grou 


Identify  eivilians. 


Execute  planned  route. 


Move  in  single  file. 


Fire  weapon  in  short  bursts. 


Understand  verbal  commands. 


Locate  assigned  areas  of  observation,  e.g.  across  the  street. 


Identify  assigned  sectors  of  observation. 


Move  according  to  directions. 


Identify  non-combatants  within  a  room. 


Identify  sector  of  responsibility. 


Execute  the  assault  as  planned. 


Communicate  enemy  location  to  team  member. 


Move  quickly  to  the  point  of  attack. 


Aim  weapon. 


Communicate  spot  reports  to  squad  leader. 


Fire  weapon  accuratel 


Coordinate  with  other  squad  members. 


Identify  covered  and  concealed  routes. 


Identify  safe  and  danger  areas. 


Assume  defensive  positions. 


Locate  support  team  positions. 


Use  hand-held  illumination  (flares). 


Identify  enemy  soldiers. 


ositions. 


Employ  tactical  hand-held  smoke  grenades. 


Maintain  position  relative  to  other  team  members. 


Identify  areas  that  mask  supporting  fires. 


Maneuver  below  windows. 


Use  flash-bang  grenades  to  help  clear  rooms. 


Take  hasty  defensive  positions. 


Engage  targets  within  a  room. 


Scan  from  side  to  side. 


Look  around  comers. 


Detenpine  other  tcam/squad  members’  positions. 


Take  position  to  one  side  of  a  doorway. 


Locate  enemy  soldiers  inside  buildings  firing  at  your  unit. 


Move  close  to  walls. 


Scan  the  room  quickly  for  hostile  combatants. 


Maneuver/move  around  obstacles. 


Use  fragmentation  grenades. 


Estimate  distances  from  self  to  a  distant  object/point. 


Maneuver  close  to  others. 


Take  a  tactical  position  within  a  room. 


Move  past  furniture  in  a  room. 


Climb  up  or  down  stairs. 


Maneuver  around  comers. 


Visually  locate  the  source  of  enemy  fire. 


Move  quickly  through  doorways. 


Distinguish  between  friendly  and  enemy  fire. _ _ 


Maneuver  past  other  personnel  within  a  room. 


Detenpine  the  direction  enemy  rounds  are  coming  from. 


Scan  vertically. 


Detenpine  the  source  of  enemy  fire  by  sound. _ 


Mean  Ratin 


2.49 


2.47 


Poorly  rated  items  included  maneuver  indoors  (close  to  others,  past  furniture,  close  to  walls, 
around  objects,  past  other  personnel,  around  comers,  through  doorways,  up  and  down  stairs),  and 
identifying  the  source  and  type  of  fire  (enemy  or  friendly),  either  by  auditory  or  visual  cues. 

Table  5. 

Summary  of  Simulator  Capability  Ratings  by  Assessment 


Assessment 

Pleban 

1999 

VE  STO 
1999 

VE  STO 
2001 

VE  STO 
2002 

V-IMTS 

2004 

SAVE 

2005 

N 

6 

18 

18 

18 

9 

26 

Mean 

Rating 

2.27 

1.90 

1.72 

2.13 

1.67 

1.90 

What  is  most  striking  about  the  ratings,  however,  as  shown  in  Table  5,  is  that  there  has 
been  no  systematic  improvement  in  the  ratings  over  a  six-year  period.  In  fact,  the  highest  ratings 
were  obtained  in  1999  (mean  =  2.27)  and  the  lowest  in  2004  (mean  =  1 .67).  Possible  reasons  for 
this  will  be  discussed  below. 

Table  6  summarizes  the  results  of  the  administration  of  the  Training  Effectiveness 
Questionnaire.  This  was  administered  to  Squad  and  Fire  Team  Leaders  in  every  assessment 
beginning  with  the  VE  STO  CE  in  1999,  and  to  Soldiers  (fire  team  members)  in  the  V-IMTS  and 
SAVE  assessments.  The  pattern  here  is  much  more  consistent,  with  the  mean  ratings  increasing 
consistently  every  year,  from  0.82  (less  than  slight  improvement)  in  1999  to  2.06  (moderate 
improvement)  in  2005.  Leaders  and  Soldiers  report  learning  different  things.  Leaders  report  the 
most  improvement  in  controlling  their  units,  assessing  the  tactical  situation,  and  communication. 
Soldiers  report  the  most  improvement  in  communication  and  planning  a  tactical  operation. 
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Figure  6.  V-IMTS  training  effectiveness  ratings  for  leaders  and  Soldiers. 

In  the  V-IMTS  assessment,  the  Squad  and  Fire  Team  Leaders  used  immersive  simulators 
(SVSI)  while  the  Soldiers  used  desktop  simulators  (SVSD).  The  training  effectiveness  ratings 
from  the  V-IMTS  assessment  are  presented  in  Figure  6.  The  mean  rating  for  training 
effectiveness  was  1.8  on  a  scale  where  1.0  equaled  slight  improvement  and  2.0  equaled  moderate 
improvement.  Interestingly,  both  the  leaders  and  the  Soldiers  reported  about  the  same  amount  of 
overall  improvement  (mean  rating  of  1.76  vs.  1.66),  although  the  skills  on  which  they  reported 
improving  the  most  differed.  The  SVSI  and  SVSD  ratings  were  positively  correlated  (r  =  .62), 
indicating  that  in  general  the  same  activities  tended  to  be  rated  similarly  on  both  simulators. 
Leaders  reported  the  greatest  improvement  on  Control  of  squad/fire  team  movement  during  the 
assault,  Assess  the  tactical  situation,  Plan  a  tactical  operation,  and  Coordinate  activities  with 
your  chain  of  command.  Soldiers  reported  the  most  improvement  on  Plan  a  tactical  operation. 
Coordinate  activities  with  your  chain  of  command,  and  Communicate  with  members  of  your  team 
or  squad.  Perhaps  because  of  the  small  numbers  involved,  the  differences  between  leader  and 
Soldier  ratings  approached  statistical  significance  for  only  one  task.  Control  of  squad/fire  team 
movement  during  the  assault 

The  2002  VE  STO  CE  included  an  objective  measure  of  unit  performance,  a  fourteen- 
item  checklist  of  unit  behaviors  that  was  scored  independently  by  three  raters  for  each  scenario. 
Possible  scores  ranged  from  a  low  of  1 .0  to  a  high  of  5.0.  Squads  received  a  mean  score  of  3.47 
on  the  first  occurrence  of  a  scenario,  and  a  mean  score  of  3.72  on  the  subsequent  occurrence  of  a 
similar  scenario.  This  provides  rare  evidence,  beyond  trainee  opinion,  that  the  training  was 
effective. 
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Other  Experiments  and  Evaluations 

Lampton,  MeDonald,  Rodriguez,  Cotton,  Morris,  Parsons,  and  Martin  (2001)  found  that 
praetice  of  a  building  seareh  task  in  VE  by  two-person  teams  results  in  improved  performance 
under  a  variety  of  training  conditions  (practice  only,  demonstration,  and  AAR). Experimental 
participants  were  college  students. 

Pleban,  Eakin,  Salter,  and  Matthews  (2001)  conducted  an  evaluation  of  the  use  of  the 
SVS™  for  training  and  assessment  of  decision  making  skills.  The  trainees  were  Army 
Lieutenants  and  Captains  playing  the  role  of  Infantry  platoon  leaders.  Role  players  and  DISAF 
were  used  to  fill  the  other  positions  required  to  conduct  the  training  scenarios.  Thus  this  was  an 
individual  training  situation.  Four  different  scenarios  were  conducted  in  a  simulated  virtual 
village:  Stability  and  Security  Operations,  Company  Assault,  Defend,  and  Secure  Village.  Role 
players  filled  the  Company  Commander,  Platoon  Sergeant,  and  Squad  Leader  positions,  and 
DISAF  was  used  to  fill  out  the  remainder  of  the  platoon.  Objective  measures  of  the  accuracy  of 
the  decisions  the  officers  made  improved  significantly  over  the  four  training  sessions  (scenarios). 
The  officers  also  reported  that  they  believed  that  their  decision  making  skills  had  improved  as  a 
result  of  the  training. 

Pleban  and  Salvetti  (2003)  evaluated  the  transfer  of  training  received  in  immersive 
simulators  (again,  the  SVSs  in  the  SBL)  to  live  simulation.  Four  Infantry  squads  participated.  In 
the  Real-World  condition,  two  squads  conducted  rehearsals  of  an  assault  in  a  real  building 
similar  to  the  one  on  which  they  would  subsequently  be  tested.  In  the  VE  condition,  two  squads 
conducted  rehearsals  in  a  VE  representation  of  the  actual  building  on  which  they  would  be 
tested.  All  squads  were  then  tested  on  two  missions  in  the  actual  building  (the  transfer  task). 

Two  observer/controllers  scored  performance  on  each  mission.  Performance  differences  between 
the  two  groups  on  the  final  two  missions  were  considered  to  be  “small  to  negligible.”  However, 
Soldier  opinions  about  the  effectiveness  of  the  VE  training  differed  depending  on  when  they 
received  it.  One-half  of  the  Soldiers  in  the  VE  condition  felt  that  it  was  just  as  effective  as  the 
real  world  for  conducting  mission  rehearsals.  However,  all  of  the  Soldiers  in  the  Real-World 
condition,  who  received  training  in  the  VE  after  completing  the  transfer  task,  felt  that  VE  was 
inferior  to  real  world  training. 

Pleban  and  Salvetti  (2003)  note  that  performance  in  the  VE  was  hampered  by  simulator 
constraints  that  made  precision  movement  in  confined  areas  and  use  of  grenades  difficult.  They 
concluded  however,  that  while  immersive  simulators  lacked  realistic  qualities  in  certain  areas, 
they  could  be  used  effectively  for  certain  types  of  training,  specifically  during  the  walk  phase  of 
training,  for  improving  decision  making,  situation  awareness,  and  communication  and 
coordination  skills. 

Loftin,  Scerbo,  McKenzie,  Catanzaro,  Bailey,  Phillips,  and  Perry  (2004)  investigated  the 
use  of  an  immersive  CAVE  (CAVE  Automatic  Virtual  Environment)  simulator  to  train 
checkpoint  operations.  The  CAVE  visual  display  consisted  of  two  10  XIO  foot  screens  each 
with  1280  X  1040  resolution.  Other  capabilities  included  a  speech  recognition  system,  position 
tracking,  and  sophisticated  avatars  for  the  computer-controlled  entities  in  the  scenarios.  Two 
groups  of  1 6  college  students  participated.  The  experimental  group  participated  in  two  45- 
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minute  shifts,  receiving  feedback  on  their  performance  between  the  shifts.  The  control  group 
participated  in  only  a  single  shift.  The  experimental  group  performed  better  on  their  second  shift 
than  on  their  first,  and  in  addition,  performed  better  on  shift  2  than  the  control  group  did  on  shift 
1 .  Both  comparisons  indicate  that  the  training  was  effective.  In  a  second  experiment,  Loftin  et  al 
replicated  the  conditions  of  the  first  experiment  using  a  monitor  and  touch  screen  in  place  of  the 
CAVE.  Although  the  number  of  participants  was  smaller,  they  were  basically  able  to  replicate 
the  findings  of  the  first  experiment:  the  simulation  (in  this  case  delivered  by  a  monitor) 
significantly  reduced  the  number  of  errors  committed  in  the  scenarios.  They  also  reported  that 
the  trainees  in  the  fully  immersive  system  (CAVE)  consistently  performed  better  than  those  in 
the  monitor  and  touch  screen  condition.  However,  they  concluded  that: 

Although  this  difference  was  found  across  groups  and  conditions,  the  magnitude 
of  the  difference  was  not  dramatic.  Thus,  the  ability  to  port  a  similar  training 
experience  to  a  less  expensive  PC  platform  without  major  performance 
differences  underscores  the  potential  for  providing  greater  access  to  this  type  of 
VE  training  in  a  much  more  cost  effective  medium(p.  12). 

Singer,  Kring,  and  Hamilton  (2006)  investigated  the  use  of  immersive  simulation  to  train 
positioning  and  maneuver  skills  in  an  urban  environment.  They  found  significant  improvements 
in  both  performance  and  the  time  required  to  perform  the  tasks.  The  basic  patterns  for  the  time 
to  perform  the  tasks  is  relatively  clear,  with  steadily  decreasing  amounts  of  time  required  for 
steadily  improving  (overall)  performance. 


Discussion  of  Immersive  Simulation  Results 


Limits  of  Experimentation 

Despite  a  decade  of  interest  and  research  and  development  activity  in  the  use  of 
immersive  simulation,  evaluations  of  the  technology  using  Soldiers  performing  Soldier  tasks 
have  been  limited.  There  are  a  number  of  reasons  for  this.  First,  experimentation  is  limited  by 
the  available  simulators,  and  the  only  facility  available  with  sufficient  simulators  to  outfit  a  full 
squad  has  been  the  SBL.  The  lack  of  availability  of  Soldiers  to  participate  in  evaluations  has 
been  a  second  constraint.  Operational  and  training  demands,  have  greatly  limited  the  number  of 
squads  and  the  amount  of  time  that  they  can  participate  in  an  evaluation.  Finally,  the  costs  of  an 
evaluation,  in  terms  of  facilities  rental,  technical  and  support  personnel,  and  travel  costs,  limit 
exercise  duration. 

SVSs  have  been  the  predominant  simulators  for  experimentation,  not  necessarily  because 
they  are  the  best  (they  might  or  might  not  be),  but  because  they  exist  in  sufficient  quantities  at  an 
Army  site.  This  means  that  to  some  extent  the  results  obtained  are  limited  to  that  simulator. 

It  is  somewhat  surprising  that  Soldier  ratings  of  SVS™  capabilities  have  not 
systematically  improved  between  1999  and  2005.  There  are  several  possible  explanations  for 
this.  One  is  that  as  capabilities  have  been  added  to  the  SVS™,  system  performance  has 
deteriorated.  While  this  might  explain  one  dip  in  the  ratings,  the  SVSs  have  been  upgraded 
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regularly  so  long-term  effects  are  unlikely.  A  second  possibility  is  that  the  training  scenarios 
have  become  more  challenging  and  complicated  in  an  attempt  to  keep  pace  with  current 
operational  conditions.  They  have  pushed  the  Soldiers  to  perform  more  complicated  tasks  in  the 
simulators,  and  therefore  they  were  more  likely  to  encounter  the  limits  of  the  simulators. 

Moving  more  of  the  action  inside  buildings,  for  example,  placed  an  emphasis  on  precise 
movement,  which  in  turn  emphasized  simulator  weaknesses  in  that  area.  The  unavailability  of 
hand  grenades  made  it  extremely  difficult  to  clear  buildings,  but  adding  hand  grenades  increased 
the  complexity  of  using  the  simulator.  It  is  also  likely  that  Soldier  ratings  of  simulator  capability 
are  strongly  influenced  by  the  amount  of  time  they  spend  in  the  simulator  acquiring  proficiency. 
This  has  not  been  controlled  over  the  years.  Finally,  the  impact  of  the  increasing  sophistication 
of  computer  and  video  games  may  have  caused  a  gradual  escalation  of  Soldier  standards  for 
simulator  performance.  The  simulator  capabilities  are  being  compared  with  increasingly  realistic 
and  sophisticated  commercial  products.  This  may  have,  in  effect,  raised  the  standards  by  which 
automated  entities  and  environments  are  judged. 

What  factors  are  responsible  for  the  increase  in  training  effectiveness  ratings?  First,  the 
complexity  and  realism  of  the  scenarios  has  improved.  Second,  the  AAR  capabilities  have 
improved.  Recall  that  the  VE  STO  program  did  not  develop  an  AAR  capability  until  2001,  and 
that  it  has  been  incrementally  improved  since.  Third,  capabilities  such  as  grenades,  smoke,  and 
improved  DISAF  behaviors  have  been  added.  However,  remember  that  the  trainer,  whether 
called  an  observer/controller  or  an  AAR  leader,  is  key  in  setting  the  effectiveness  of  the  training. 

The  results  of  the  V-IMTS  assessment,  combined  with  the  results  of  the  Loftin  et  al 
(2004)  experiment,  raise  the  question  of  how  realistic  simulators  need  to  be  in  order  to  be 
effective.  An  interesting  and  unexpected  aspect  of  the  V-IMTS  results  is  the  lack  of  a  significant 
difference  in  rated  training  effectiveness  between  the  Soldiers  trained  in  the  desktop  simulators 
and  the  leaders  trained  in  the  immersive  simulators.  Admittedly,  they  were  performing  and 
learning  different  tasks,  but  nevertheless  it  seems  to  suggest  that  the  features  of  a  fully  immersive 
simulator  might  not  produce  a  training  benefit  that  justifies  the  cost  premium  over  a  simple 
desktop  computer.  Loftin  et  al  (2004)  found  small  differences  in  effectiveness  between 
immersive  and  desktop  simulators,  and  questioned  the  effectiveness  of  the  immersive  simulator. 

Research  on  Simulator  Fidelity 

Other  research  addressing  the  question  of  interface  fidelity  on  training  effectiveness  is 
limited.  The  basic  concept  is  that  immersive  VE  systems,  as  compared  to  desktop  VE  systems, 
provide  the  trainee  with  more  information  about  their  orientation  in  and  movement  through 
physical  space.  Systems  with  head-tracked  HMDs  provide  vestibular  and  kinesthetic  cues  about 
changes  in  direction  of  movement  and  viewing  direction.  Systems  with  a  walking  mechanism, 
such  as  a  treadmill,  or  which  use  sensors  to  implement  walking  in  place,  provide  kinesthetic  cues 
about  distance  traveled.  Systems  with  wide  FOV  displays  provide  visual  cues  in  the  periphery  of 
the  visual  field  about  distance  and  speed  of  movement.  If  these  factors  make  a  difference  in 
effectiveness,  they  would  most  likely  affect  learning  routes  through  and  the  configuration  of 
physical  spaces,  an  area  of  research  known  as  wayfmding.  Researchers  in  this  area  make  a 
distinction  between  route  knowledge  (which  allows  one  to  successfully  move  from  one  known 
point  to  another  known  point  along  a  specific  route,  using  landmarks  and  waypoints,  but  does  not 
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allow  for  deviations  from  the  route),  and  configuration  or  survey  knowledge  (which  enables  uses 
to  create  a  map  of  an  area  or  create  new  routes). 

As  background,  it  should  be  noted  that  the  question  of  whether  VR  systems  can  be  used 
to  train  configuration  and  route  knowledge  is  not  in  dispute.  A  large  number  of  experiments 
have  shown  that  they  can  (e.g.,  Witmer,  Bailey  and  Knerr,  1995;  Bliss,  Tidwell,  and  Guest, 

1997;  Tate,  Siebert,  and  King,  1997).  The  question  is  whether  more  realistic  and  sophisticated 
interfaces  enhance  that  learning. 

Grant  and  Magee  (1997)  compared  a  walking  in  place  interface  with  a  joystick  in  an 
experiment  investigating  the  acquisition  and  transfer  of  spatial  knowledge.  All  participants 
viewed  the  VE  through  a  head-tracked  HMD.  The  participants'  task  was  to  learn  the  location  of  a 
number  of  landmark  objects  in  a  science  museum.  During  a  transfer  test  in  the  actual  building, 
the  walking  simulator  group  walked  significantly  less  distance  to  find  all  of  the  landmarks  than 
did  the  joystick  trained  group,  but  they  did  not  find  them  significantly  faster.  Grant  and  Magee 
claim  this  as  evidence  that  there  is  something  learned  from  the  walking  interface  in  addition  to 
the  visual  information  that  can  be  used  for  spatial  navigation. 

Singer,  Allen,  McDonald,  and  Gildea  (1997)  found  that  training  a  Hi-VE  interface  (head- 
linked  stereoscopic  HMD  and  treadmill)  produced  better  configuration  knowledge  and  more 
rapid  response  time  than  a  Low-VE  interface  (non-head-linked  stereoscopic  HMD  and  joystick) 
in  a  terrain  learning  task.  Students  trained  using  the  Hi-VE  interface  also  did  better  than  those 
trained  using  topological  maps. 

Waller,  Hunt,  and  Knapp  (1998)  compared  blindfolded  maze-walking  performance  of 
groups  trained  with  immersive  VE  (HMD  with  no  head-tracking  plus  joystick),  desktop  VE 
(monitor  plus  joystick)  and  map,  among  other  conditions.  While  the  performance  of  all  groups 
improved,  the  map  condition  was  superior  to  the  immersive  VE  and  desktop  VE  conditions, 
which  did  not  differ.  However,  their  immersive  VE  condition  provided  no  additional  kinesthetic 
or  vestibular  cues  relative  to  the  desktop  VE  condition. 

Jacquet  (2002)  found  no  significant  differences  between  groups  trained  to  perform  tank 
turret  troubleshooting  tasks  using  immersive  (HMD  and  3D  mouse)  and  desktop  (two  monitors 
and  a  mouse)  versions  of  an  Ml  tank  turret  simulator. 

Lathrop  and  Kaiser  (2005)  compared  immersive  and  desktop  interfaces  in  terms  of  search 
performance  and  maintenance  of  orientation.  Immersive  users  wore  a  head-tracked  HMD, 
changed  direction  of  movement  by  rotating  while  seated  in  a  chair,  and  clicked  a  button  on  a 
hand-held  mouse  to  move  forward  or  backward.  Desktop  users  viewed  a  Monitor  and  used  the 
hand-held  mouse  to  both  turn  and  move  forward  or  backward.  Immersive  users  were  able  to 
locate  target  objects  in  a  complex  environment  more  quickly  than  desktop  users,  but  did  not 
develop  better  configuration  knowledge  of  the  space  they  searched. 

Singer,  Ehrlich,  Cinq-Mars,  and  Pappin  (1995),  investigating  the  effects  of  head-coupling 
and  stereoscopic  displays  on  the  performance  of  simple  tasks  in  VE,  found  that  those  two 
variables  had  a  significant  effect  only  on  distance  estimation  at  short  distanees  (10  feet  or  less). 
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In  summary,  it  appears  that  head-tracked  visual  displays,  body  controlled  movement,  or  a 
combination  of  the  two  can  improve  the  performance  of  spatially-oriented  tasks  and  acquisition 
of  spatial  knowledge,  but  this  difference,  based  on  a  limited  number  of  experiments,  does  not 
appear  to  be  large. 
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The  Use  of  Computer  Games  for  Training 


Over  the  past  ten  years,  there  has  been  increasing  interest  in  the  use  of  computer  games, 
played  on  either  a  PC  or  a  game  console,  such  as  the  X-Box,  Playstation,  or  Gameboy,  for  Army 
training.  Once  these  games  were  believed  to  be  visually  attractive,  but  too  limited  in  terms  of  the 
actions  they  permitted  the  players  and  those  of  the  simulated  entities,  and  too  unrealistic  in  terms 
of  how  players  made  those  actions  (e.g.,  using  a  game  controller,  joystick,  or  mouse),  to  be 
considered  as  a  means  of  training.  Today,  games  are  being  seriously  considered  for  a  variety  of 
reasons. 

First,  it  is  assumed  that  most  Soldiers  play  computer  games.  Two  additional  assumptions 
follow  from  this  one.  The  first  is  that  Soldiers  are  motivated  to  play  games  designed  to  train, 
even  to  the  extent  that  they  will  do  so  in  their  off-duty  hours.  The  second  is  that  Soldiers  are 
skilled  in  plajdng  games,  and  therefore  they  need  little  introductory  train-up  or  preparation,  such 
as  game  concepts  or  learning  how  to  use  game  controllers,  prior  to  using  these  games  for 
training. 

Second,  game  consoles  and  PCs,  particularly  laptop  PCs,  are  readily  transportable  and 
deployable.  They  are  also  low  cost  relative  to  special  purpose  simulators. 

Third,  many  games  can  be  networked  to  provided  collective  training.  Some  games 
(massively  multiplayer  games)  can  include  hundreds  of  simultaneous  participants. 

Finally,  the  use  of  computer  games  provides  a  way  to  capitalize  on  the  investments 
already  made  by  game  developers.  This  can  translate  into  low  per  copy  acquisition  costs, 
particularly  if  the  game  can  be  used  as  is. 

The  issue  of  training  effectiveness,  however,  is  the  most  important  consideration.  If 
games  do  not  provide  effective  training,  then  the  reasons  given  for  their  use  do  not  matter.  What 
evidence  is  there  that  computer  games  can  provide  effective  training? 

Hayes  (2005)  conducted  an  extensive  review  of  the  use  of  games  for  education  and 
training.  Of  270  documents  he  identified  in  a  literature  search,  only  48  reported  evaluations  in 
which  empirical  data  on  effectiveness  was  obtained.  Of  those,  six  used  military  personnel  as 
trainees.  Skills  trained  in  those  six  experiments  included  periscope  skills;  chemical,  biological, 
and  radiological  defense  principles;  and  technical,  electrical,  or  electronics  skills  and 
terminology.  Three  experiments  showed  better  learning  or  retention  with  the  game,  one  worse, 
one  no  difference,  and  one  showed  transfer  of  the  skills  learned  in  the  game  but  had  no  control 
group  for  comparison.  Hayes’  review  showed  that  games  can  provide  effective  training  for  a 
variety  of  training  audiences  and  a  variety  of  skills.  However,  he  concluded  that  there  is  no 
evidence  that  instructional  games  are  more  effective  than  other  well-designed  instructional 
activities. 

While  a  few  of  the  evaluations  reviewed  by  Hayes  represent  military  tasks,  fewer  address 
the  issue  we  are  most  concerned  with.  Soldier  training  of  combat  tasks. 
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DeBrine  and  Morrow  (2000)  demonstrated  that  a  computer  off-the-shelf  (COTS) 
computer  game  (Quake  III  Arena™)  could  be  modified  to  support  conduct  of  a  walkthrough  of  a 
real  building.  It  took  about  300  hours  of  labor  for  unskilled  programmers  to  modify  the  game 
code  and  implement  the  building  model.  No  try-outs  or  evaluations  of  the  resulting  game  were 
conducted. 

Beal  and  Christ  (2004)  conducted  an  evaluation  of  Full  Spectrum  Command  (FSC),  a  PC 
game  designed  specifically  for  use  in  the  Infantry  Captains  Career  Course  (ICCC).  Fifty-four 
ICCC  students  participated.  One-half  of  the  students  participated  in  their  normal  course  work. 
The  rest  did  that  and  in  addition  played  FSC  for  an  average  of  4.8  hours  (in  addition  to  4.8  hours 
of  demonstration  and  train-up).  The  decision  making  of  officers  in  both  groups  was  then 
assessed  as  they  conducted  individual  exercises  using  the  JANUS  Simulation.  No  differences  in 
decision  making  performance  during  the  JANUS  exercises  were  found.  Overall,  the  trainees 
reported  highly  positive  opinions  about  the  overall  training  value  of  FSC.  They  reported  that  the 
scenarios  were  challenging,  the  realism  of  the  battlefield  events  and  actions  was  quite  good,  and 
playing  FSC  provided  practice  in  many  of  the  tasks  that  were  relevant  to  course  objectives.  On 
the  other  hand,  the  majority  indicated  that  they  had  to  focus  their  attention  on  the  game’s  display 
and  control  devices,  rather  than  on  the  experiences  created  by  the  game. 

Beal  and  Christ  (2004)  believe  that  they  failed  to  find  a  difference  between  the  two 
groups  in  performance  on  the  JANUS  exercise  because  of  limited  time  available  to  play  FSC,  the 
lack  of  controlled  procedures  for  use  of  FSC  (e.g.,  trainee  selection  of  scenarios),  and  the  lack  of 
any  feedback  or  evaluation  of  decision  making  performance  during  or  after  the  FSC  scenarios. 
While  an  AAR  capability  was  built  into  FSC,  it  was  not  used  due  to  operational  constraints. 

Belanich,  Sibley,  and  Orvis  (2004)  conducted  an  experiment  using  the  “America’s 
Army”  game,  a  first  person  shooter  designed  as  a  recruiting  tool.  Twenty  delayed  entry  recruits 
and  one  Reserve  Officer  Training  Corps  cadet  played  the  basic  training  section  of  the  game, 
consisting  of  Army  background  information,  marksmanship  training,  obstacle  course,  weapons 
familiarization,  and  a  MOUT  mission.  After  playing  the  game,  the  participants  answered  a  series 
of  multiple  choice  questions  about  information  presented  in  the  game.  The  participants 
remembered  procedures  better  than  facts,  and  information  presented  in  graphics  and  spoken  text 
better  than  information  presented  in  written  text  (perhaps  because  the  written  text  could  be 
skipped  over  without  reading).  They  also  recalled  information  relevant  to  progressing  through 
the  game  better  than  irrelevant  information. 

Diller,  Roberts,  Blankenship,  and  Nielsen,  (2004)  evaluated  the  DARWARS  AMBUSH! 
at  Fort  Lewis,  WA,  in  May  2004.  AMBUSH!  is  a  computer-based  collective  training  game  that 
lets  units  experience  and  respond  to  ambush  situations.  A  total  of  18  participants  (16  enlisted 
personnel  and  two  officers)  used  the  system  in  three  groups.  Each  group  participated  for  about 
three  hours,  to  include  a  training  session  and  the  conduct  of  three  missions.  At  the  conclusion  of 
the  training,  participants  rated  their  overall  satisfaction  with  the  system  as  a  convoy  operations 
training  tool  as  5.88,  on  a  scale  of  1  (low)  to  7  (high).  All  participants  gave  ratings  at  or  above 
the  midpoint  of  the  scale  (4).  Participants  were  even  more  positive  about  using  the  system  as  a 
tactics,  techniques,  and  procedures  training  tool  (mean  =  6.36).  Trainee  performance  was  not 
assessed. 
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Beal  and  Christ  (2005)  conducted  an  assessment  of  the  Rapid  Decision  Trainer  (RDT). 
The  RDT  is  a  PC-based  simulation  built  on  game  engine  software.  It  was  designed  to  prepare 
Infantry  lieutenants  to  serve  as  platoon  leaders  during  the  Infantry  Officer  Basic  Course  (lOBC) 
platoon  Live  Fire  Exercise  (LFX).  It  requires  the  trainee  to  conduct  mission  analysis  and 
planning  and  to  make  decisions  in  response  to  conditions  that  emerge  during  the  execution  of  the 
simulated  mission.  This  assessment  used  a  single  RDT  scenario,  which  closely  resembled  the 
terrain  and  battle  conditions  that  occur  during  the  lOBC  LFX.  Twenty  lieutenants  participated  in 
the  RDT  scenario  as  two  person  teams,  a  platoon  leader  and  an  observer,  who  rotated  positions. 
Nineteen  lieutenants  participated  as  a  large  group,  some  of  whom  rotated  through  the  platoon 
leader  position  while  the  others  observed  the  exercise.  Questionnaires  were  used  to  obtain 
feedback  Ifom  the  trainees  after  the  RDT  training  but  both  before  and  after  the  LFX.  No 
performance  data  was  collected.  Trainees  indicated  that  the  RDT: 

•  Permitted  training  and  rehearsal  of  the  types  of  decisions  a  platoon  leader  must  make 
during  a  LFX  (85%  of  ratings  in  highest  three  [of  seven]  categories) 

•  Provided  meaningful  practice  for  exercising  command  and  control  of  platoon 
operations  (82%) 

•  Accurately  simulated  tasks  and  conditions  specified  in  current  platoon-level  battle 
drills  (80%) 

•  Had  a  valuable  impact  on  preparing  them  for  leading  a  platoon  in  a  unit  (77%) 

•  Provided  meaningful  practice  for  planning  a  platoon  mission  (72%) 

Despite  concluding  that  the  RDT  had  training  value,  the  authors  also  provided  a  number  of 
cautions:  the  sample  size  was  small,  no  performance  measures  were  available,  training  time  on 
the  RDT  was  limited,  only  one  scenario  was  available,  and  the  simulation  was  still  under 
development. 

Nolan  and  Jones  (2005)  assessed  the  use  of  Delta  Force:  Black  Hawk  Down  -  Team 
Sabre™  to  train  lOBC  students  to  perform  the  React  to  Contact  Battle  Drill.  In  this  case,  full 
squads  at  individual  computers  were  networked  together  to  conduct  the  training  scenarios.  Two 
squads  each  completed  two  eight-hour  training  sessions  (completing  six  or  seven  missions)  using 
the  game  in  addition  to  their  regular  training.  Two  control  squads  received  only  their  regular 
training.  All  squads  then  conducted  a  live  exercise,  and  completed  questionnaires.  All 
participants  in  the  game  condition  rated  the  training  effectiveness  of  the  game  as  “good”  or 
better.  All  game  participants  also  reported  that  the  game  had  a  “positive”  or  “very  positive” 
effect  on  their  performance  on  the  subsequent  live  exercise.  No  performance  data  on  the  live 
exercise  were  obtained. 


Issues  in  Game-Based  Training 

ARI  hosted  a  symposium  on  the  use  of  PC-based  simulations  and  gaming  for  military 
training  in  November  2003  (Belanich,  Mullin,  and  Dressel,  2004).  Approximately  50  researchers 
and  trainers  participated.  Beal  (2005)  documented  lessons  learned  and  issues  based  on  his 
conduct  of  evaluations  of  the  RDT,  Full  Spectrum  Command,  and  FSW.  There  is  considerable 
similarity  between  the  symposium  conclusions  and  Beal’s  lessons  learned. 
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First  and  most  important,  the  consensus  is  that  games  can  be  effective  for  training,  but  the 
empirical  evidence  to  support  this  is  lacking.  In  particular,  there  is  a  belief  that  games  can  be 
effective  in  training  cognitive  skills  prior  to  their  application  in  field  exercises.  In  other  words, 
games  could  bridge  the  gap  between  classroom  training  and  field  exercises. 

Instructors  are  believed  to  be  highly  important  to  the  successful  use  of  games  for  training. 
Beal  (2005)  gives  two  general  areas  of  instructor  support.  The  first  is  in  explaining  the  purpose 
of  the  game,  demonstrating  game  functions,  and  explaining  game  limitations  and  shortfalls.  The 
second  is  in  providing  coaching  and  feedback,  to  include  the  conduct  of  AARs.  Feedback  to  the 
trainee  by  the  instructor  was  the  most  frequently  mentioned  instructional  techniques  for  use  with 
games.  This  feedback  can  range  from  immediate  feedback  during  the  game  or  scenario  to  an 
AAR  at  its  conclusion.  Feedback  should  both  highlight  positive  lessons  learned  and  diminish  the 
possibility  that  the  trainees  will  learn  suspect  behaviors  that  may  enhance  success  in  the  game, 
but  not  in  the  real  world. 

Also  important  is  the  need  for  game  design  to  be  based  on  training  objectives.  Clear 
training  objectives  and  follow-through  by  instructors  are  important  for  successful 
implementation. 

Data  on  game  playing 

Both  the  ARI  symposium  and  Beal  (2005)  called  into  question  the  assumption  that  all 
Anny  trainees  are  skilled  game  players.  The  attractiveness  of  these  games,  in  addition  to 
presumed  training  effectiveness,  is  that  Soldiers  are  assumed  to  already  be  familiar  with  them, 
and  motivated  to  train  with  them,  even  in  off-duty  hours.  Are  these  assumptions  justified? 
Symposium  participants  considered  the  time  required  to  learn  to  play  a  game  is  a  barrier  to 
successful  implementation. 

Roberts,  Foehr,  and  Rideout  (2005)  conducted  an  extensive  survey  of  American  youth 
and  media  in  2004.  They  found  that  8-18  year  olds  spent  an  average  of  19  minutes  per  day 
playing  PC  games  and  49  minutes  playing  console  and  handheld  games.  Fifty-nine  percent  (68% 
of  males  and  51%  of  females)  play  some  type  of  interactive  games  on  any  given  day.  These 
results  seem  to  be  fairly  consistent  across  race  and  parent’s  education  and  income.  However,  it 
cannot  be  detenuined  Ifom  the  report  what  percentage  of  youth  rarely  or  never  play  interactive 
games. 


No  recent  large-scale  surveys  of  Soldier  game  playing  have  been  conducted.  However, 
Soldiers  participating  in  evaluations  of  simulations  and  PC  games  have  been  asked  about  their 
game-playing  experience.  Soldiers  in  the  V-IMTS  evaluation  described  earlier  reported  a  mean 
of  9.5  hours  (median  6.5  hours)  per  week  playing  computer  or  video  games.  A  sample  of  27 
Infantry  Soldiers  from  a  separate  assessment  conducted  in  FY  2004  (Knerr,  Garrity,  and 
Lampton,  2004)  reported  a  mean  of  8.5  hours  (median  5.0  hours)  per  week  playing  computer  or 
video  games.  However,  7%  and  22%  of  the  Soldiers  in  these  samples,  respectively,  did  not  play 
video  games.  Beal  and  Christ  (2005)  found  that  30%  of  their  sample  of  39  Infantry  lieutenants 
involved  in  the  evaluation  of  the  RDT  did  not  play  video  games,  and  51%  described  themselves 
as  Novice  PC  game  players.  Only  13%  reported  playing  more  than  two  hours  per  week.  In  their 
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evaluation  of  the  FSC  game,  Beal  and  Christ  (2004)  found  that  57%  of  their  sample  of  54 
Infantry  lieutenants  and  captains  did  not  play  computer  games.  Diller,  Roberts,  Blankenship, 
and  Nielsen  (2004),  when  evaluating  the  game  DARWARS  Ambush!,  found  that  14  of  their  18 
participants  reported  playing  computer  games  for  at  least  one  hour  per  week  (mean  =  12.9  hours 
per  week).  Their  sample  included  2  officers  and  16  enlisted  personnel,  three  of  whom  were 
women.  Nolan  and  Jones  (2005)  reported  that  10  of  the  41  lOBC  students  in  their  experiment 
had  played  no  computer  games  in  the  previous  two  years,  and  only  1 5  had  played  weekly  or 
more  frequently. 

The  results  involving  enlisted  personnel  seem  fairly  consistent  with  those  of  the  Roberts, 
Foehr,  and  Rideout  (2005)  survey  data  reported  above.  The  results  involving  junior  officers 
seem  very  different.  Whether  this  is  a  result  of  differences  in  age,  education,  social  factors,  or  the 
particular  questions  asked  is  not  clear.  It  is  safe  to  assume  that  a  large  portion  of  our  potential 
trainees  are  familiar  with  computer  or  video  games.  However,  we  cannot  assume  that  all  of  them 
are  skilled  or  enthusiastic  game  players.  This  does  not  mean  that  games  could  not  provide 
effective  training  for  them,  but  it  does  indicate  a  need  to  provide  adequate  training  in  playing  the 
game  before  training  begins  in  earnest.  Beal  (2005)  found  that  leaders  differed  in  their 
proficiency  with  games  and  personal  computers.  More  than  70%  of  the  participants  in  the  FSC 
evaluation  reported  that  they  had  to  focus  on  computer  control  devices  and  functions  rather  than 
the  experiences  created  by  the  simulation.  He  reported  finding  similar  results  in  other 
evaluations.  Experimenters  tended  to  overestimate  trainee  proficiency  and  underestimate 
training  time. 

A  related  question  is  whether  Soldiers  have  access  to  computers  to  use  for  training  in 
their  off-duty  hours.  The  ARI  Army  Personnel  Survey  Office  conducted  a  survey  in  the  fall  of 
2002  and  found  that  99.1%  of  all  Army  officers  and  87.4%  of  all  enlisted  personnel  reported 
having  access  to  the  internet,  while  98.7%  and  84.1%,  respectively,  reported  having  access  to  a 
PC.  The  group  with  the  lowest  access  was  junior  enlisted  personnel,  with  only  80.4%  having 
access  to  the  internet  and  74.8%  having  access  to  a  PC.  These  data  may  have  changed,  but  no 
recent  survey  data  are  available.  Moreover,  having  a  computer  and  internet  access  in  the 
barracks  or  residence  does  not  guarantee  that  those  resources  can  be  used  by  the  Soldier  for 
training  in  off-duty  hours. 
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Simulator  Sickness 


Simulator  sickness  refers  to  unwanted  side  effects  and  aftereffects  that  may  result  from 
using  simulators,  but  does  not  result  from  similar  use  of  the  actual  equipment.  Major  symptoms 
include  nausea,  dizziness,  and  headache  or  eyestrain.  Simulator  sickness  is  closely  related  to,  but 
nevertheless  different  from,  forms  of  motion  sickness  such  as  sea  sickness,  car  sickness,  and  air 
sickness.  Simulator  sickness  is  a  concern  because  it  can  affect  the  well  being  of  the  trainee  and 
reduce  training  effectiveness,  even  if  it  does  not  produce  severe  symptoms.  Discomfort  in  the 
simulator  may  distract  the  trainee.  It  may  also  cause  trainees  to  adopt  behaviors  that  reduce 
discomfort  in  the  simulator,  but  will  be  detrimental  if  transferred  to  the  aetual  task  (such  as 
reducing  side-to-side  scanning).  Aftereffects  involving  the  sense  of  balance  or  visual  flashbacks 
could  impair  the  trainees'  ability  to  drive  safety  or  perform  skilled  motor  tasks  after  leaving  the 
simulator.  The  training  value  of  a  simulator  is  also  reduced  if  simulator  sickness  causes  a 
reduction  in  the  frequency  or  duration  of  its  use. 

Experts  do  not  agree  about  the  causes  of  simulator  sickness  (Johnson,  2005;  Kolasinski, 
1995).  However,  according  to  the  most  popular  theory,  simulator  sickness  is  thought  to  result,  at 
least  in  part,  because  simulated  movement  results  in  a  conflict  between  the  body's  mechanical 
systems  and  visual  systems  for  sensing  movement.  Normally,  our  sense  of  balance  is  maintained 
by  a  complex  interaction  of  several  senses  including:  the  eyes,  which  monitor  where  the  body  is 
in  space  and  direction  of  motion;  the  inner  ears,  which  monitor  the  directions  of  motion;  the  skin 
pressure  receptors  such  as  in  the  feet  and  seat,  which  indicate  what  part  of  the  body  is  down  and 
touching  the  ground;  and  the  muscle  and  joint  sensory  receptors,  which  indicate  what  parts  of  the 
body  are  moving.  When  the  relationships  among  these  senses  differ  from  our  normal  experience 
(as  when  our  eyes  perceive  us  moving  down  a  corridor  while  our  other  senses  indicate  that  we 
are  standing  still),  discomfort  results. 

Kolasinski  (1995)  has  documented  a  variety  of  factors  that  affect  the  incidence  and 
severity  of  simulator  sickness.  These  are  shown  in  Table  7.  It  is  not  necessary  to  review  these  in 
detail.  However,  it  is  important  to  recognize  that  the  extent  to  which  a  trainee  will  experience 
simulator  sickness  symptoms  will  depend  on  their  individual  characteristics,  the  characteristics 
of  the  simulator,  and  the  characteristics  of  the  task  they  are  trying  to  perform. 

Simulator  sickness  is  usually  measured  by  using  questionnaires.  A  commonly  used 
questionnaire  is  the  Simulator  Sickness  Questionnaire  (SSQ),  (Kennedy,  Lane,  Berbaum,  & 
Lilienthal,  1993).  The  SSQ  consists  of  16  symptoms,  each  of  which  is  rated  by  the  simulator  user 
on  a  4-point  scale  (0=none,  l=slight,  2=moderate,  3=severe).  These  ratings  form  the  basis  for 
three  subscale  scores  (Nausea,  Oculomotor  Discomfort,  and  Disorientation)  as  well  as  a  Total 
Severity  score.  The  symptoms  making  up  the  three  factor  scores  are  as  follows:  Nausea  -  general 
discomfort,  increased  salivation,  sweating,  nausea,  difficulty  concentrating,  stomach  awareness, 
and  burping;  Oculomotor  -  general  discomfort,  fatigue,  headache,  eyestrain,  difficulty  focusing, 
difficulty  concentrating,  and  blurred  vision;  and  Disorientation  -  difficulty  focusing,  nausea, 
fullness  of  head,  blurred  vision,  dizzy  (eyes  open),  dizzy  (eyes  closed),  and  vertigo.  One 
complication  is  that  these  symptoms  as  not  unique  to  simulator  sickness.  They  can  be  caused  by 
a  variety  of  factors,  such  as  insufficient  sleep,  cold,  flu,  or  alcohol  consumption. 
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Table  7 

Potential  Factors  Associated  with  Simulator  Sickness  in  Virtual  Environments 


Individual 

Simulator 

Task 

• 

age 

binocular  viewing 

•  altitude  above  terrain 

• 

concentration  level 

calibration 

•  degree  of  control 

• 

ethnicity 

color 

•  duration 

• 

experience  with  real-world 

contrast 

•  global  visual  flow 

task 

field  of  view 

•  head  movements 

• 

experience  with  simulator 

flicker 

•  luminance  level 

(adaptation) 

inter-pupillary  distance 

•  method  of  movement 

• 

flicker  fusion  frequency 

motion  platform 

•  rate  of  linear  or  rotational 

threshold 

phosphor  lag 

acceleration 

• 

gender 

position-tracking  error 

•  self-movement  speed 

• 

illness  and  personal 

refresh  rate 

•  sitting  versus  standing 

characteristics 

resolution 

•  type  of  application 

• 

mental  rotation  ability 

scene  content 

•  unusual  maneuvers 

• 

perceptual  style 

time  lag  (transport  delay) 

•  vection 

• 

postural  stability 

update  rate  (frame  rate) 
viewing  region 

Is  simulator  sickness  really  a  problem  for  users  of  immersive  simulators?  Knerr  et  al 
(1998)  reported  that  simulator  sickness  symptoms  among  college  students  who  participated  in  a 
series  of  VE  laboratory  experiments  were  sufficiently  severe  that  5.6%  (29/517)  withdrew  from 
the  experiments.  Fourteen  (4.1%  percent)  of  the  338  participants  who  were  specifically  asked 
indicated  vomiting. 

Soldiers  have  typically  reported  fewer  and  less  severe  symptoms.  Lockheed  Martin 
(1997)  reported  low  levels  of  symptoms  among  the  Soldiers  during  the  DWN  engineering 
experiments  (about  one  “slight”  symptom  for  every  four  Soldiers).  Scores  for  the  DWN  ERT 
experiments  were  higher.  For  three  of  the  VICS,  Soldiers  reported  symptoms  equivalent  to  two 
slight  symptoms  at  the  end  of  each  day.  However,  Soldiers  who  used  VIC  Golf  reported  the 
equivalent  of  five  slight  symptoms.  Recall  that  VIC  Golf  used  a  combination  of  HMD  and  ODT. 

In  the  2001  VE  STO  CE,  total  symptom  scores  were  slightly  higher  at  the  end  of  the  day 
than  at  the  start  (0.67  vs.  0.31)  but  the  difference  was  slight,  equating  to  an  increase  of  about  one 
slight  symptom  for  every  three  soldiers.  The  most  frequently  reported  symptoms  at  the  end  of 
the  day  were  fatigue  and  eyestrain,  both  with  mean  symptom  scores  of  0.17,  a  level  of 
occurrence  equal  to  about  one  slight  symptom  reported  for  every  six  soldiers. 

In  the  2002  VE  STO  CE,  the  occurrence  of  simulator  sickness  symptoms  was  very  low, 
averaging  the  equivalent  of  1 .36  slight  symptoms  per  soldier  at  the  end  of  the  day,  compared 
with  1 . 1 6  at  the  beginning.  The  largest  increases  were  in  the  symptoms  headache  and  eyestrain 
(+.17  each). 

In  V-IMTS  evaluation,  there  was  an  increase  in  the  overall  frequency  and  severity  of 
symptoms  following  simulator  use,  but  the  increase  was  slight,  averaging  about  the  equivalent  of 
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one-half  (0.55)  a  unit  increase  in  one  symptom.  The  increase  was  larger  for  Solders  in  the 
immersive  simulators  (1.00)  than  for  Soldiers  using  the  desktop  simulators.  (0.32). 

The  low  level  of  occurrence  of  symptoms  likely  results  from  the  use  of  short  scenarios 
(approximately  20  minutes  each)  separated  by  non-immersive  activities,  and  the  use  of  rear- 
projection  displays  in  the  SVSs.  Simulator  sickness  does  not  seem  to  be  an  obstacle  to  longer- 
duration  training  scenarios. 

There  was  one  additional  assessment  that  obtained  information  about  Soldier  simulator 
sickness  using  immersive  simulators.  It  has  not  been  discussed  previously  because  it  was 
concerned  strictly  with  simulator  capability,  not  training  effectiveness.  It  involved  three  squads 
conducting  exercises  with  immersive  and  desktop  SVSs,  and  three  different  versions  of  wearable 
simulators  which  used  HMDs.  All  of  the  HMDs  permitted  peripheral  views  of  the  real  world 
around  the  sides  and  bottom.  The  SVSs  and  two  of  the  three  wearable  simulators  produced 
syiuptom  scores  in  the  1.19-  1 .56  range,  but  the  third  wearable  simulator  produced  a  means 
symptom  score  of  4.33.  The  highest  scoring  symptoms  for  that  system  were  eyestrain  (.58), 
general  discomfort  (.44),  headache  (.47),  difficulty  focusing  (.35)  and  blurred  vision  (.26).  This 
suggests  that  the  visual  display  system  was  a  causal  factor.  The  same  pattern  remained  when  we 
looked  at  the  scores  for  three  different  dimensions  of  simulator  sickness.  Nausea,  Oculomotor 
Discomfort,  and  Disorientation:  one  simulator  produced  higher  scores  than  the  others,  which  did 
not  differ  among  themselves.  This  was  probably  a  result  of  the  particular  HMD  it  used.  It  had  a 
very  small  exit  pupil,  which  meant  that  it  had  to  be  precisely  positioned  on  the  head.  Head 
movements  could  cause  a  temporary  loss  of  the  image.  It  also  had  an  adjustment  for  inter¬ 
pupillary  distance,  which  each  Soldier  had  to  make.  If  this  was  not  done  properly,  the  images 
seen  by  the  two  eyes  would  not  fuse,  and  eyestrain  would  result.  There  was  no  way  to  check  to 
see  it  the  adjustment  had  been  made  properly. 

What  can  we  conclude?  First,  consider  that  in  all  situations  involving  Soldiers,  exposures 
were  short,  usually  under  30  minutes  per  exercise  (although  Soldiers  participated  in  multiple 
exercises  per  day).  Longer  exercises,  particularly  with  Soldiers  who  have  little  experience  with 
the  simulators,  may  cause  more  problems.  Second,  with  a  few  exceptions,  the  Soldiers  used  the 
SVS™  immersive  simulator,  which  provides  view  of  the  real  world  (the  floor  and  walls  of  the 
simulator  enclosure)  while  viewing  the  simulation  on  a  rear-projection  display.  Other  display 
systems,  such  as  wide  FOV  projection  systems  or  HMDs  which  completely  obscure  the  view  of 
the  real  world  are  more  likely  to  cause  problems. 

Simulator  sickness  should  be  viewed  as  a  real  but  manageable  problem. 
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Discussion 


Let  us  review  some  of  the  results  presented  earlier.  A  number  of  reviews  have  concluded 
that  simulation  is  a  cost-effective  means  of  training.  It  is  not  necessarily  more  effective  or 
efficient  than  the  real  world  or  use  of  the  actual  equipment,  but  it  typically  costs  less.  Simulators 
employing  older  technology,  such  as  the  Squad  Engagement  Training  System  and  the 
Engagement  Skills  Trainer  2000,  have  proven  to  be  effective  for  leader  and  weapons  training. 
SME  ratings  of  virtual  simulators  have  tended  to  indicate  potential,  if  not  the  actual  effectiveness 
of  the  simulators. 

With  regard  to  individual  training,  there  seems  to  be  a  consistent  finding  that  training  in 
virtual  simulations  can  be  effective.  Whether  the  objective  is  wayfinding  and  navigation, 
checkpoint  operations,  tactical  positioning  and  maneuver,  or  squad  leader  decision  making,  there 
is  ample  objective  evidence  that  trainee  skills  improve  with  practice  in  the  simulation. 

The  results  with  regard  to  collective  training  are  more  ambiguous.  Trainee  ratings 
indicate  that  they  can  perform  a  wide  variety  of  individual  tasks  in  the  simulators  at  a  level  of 
“good”  or  better,  which  is  a  necessary  but  not  sufficient  condition  for  effective  training.  Trainee 
ratings  have  indicated  that  they  believe  that  their  performance  improved  as  a  result  of  training  in 
the  virtual  simulators.  Observer/Controller  ratings,  while  limited,  have  indicated  that  leader/unit 
performance  improved  as  a  result  of  training  in  virtual  simulators. 

Whether  or  not  one  concludes  that  immersive  simulations  are  effective  for  collective 
training  of  dismounted  Soldiers  depends  largely  on  the  standard  of  proof  employed.  The  “gold 
standard”  would  be  an  improvement  in  objectively  measured  performance  in  actual  combat  or 
live  exercises,  in  controlled  experiments  so  that  the  improvement  can  be  shown  to  result  from  the 
use  of  the  immersive  simulation  to  the  exclusion  of  other  factors,  and  a  sufficient  number  of 
units  trained  to  insure  confidence  that  the  results  do  not  result  from  chance.  Unfortunately,  there 
is  no  evidence  that  meets  this  standard.  Conversely,  there  is  no  evidence  that  immersive 
simulation  could  not  meet  this  standard.  Such  an  evaluation  has  not  been  conducted. 

Without  the  evidence  that  a  “gold  standard”  evaluation  would  provide,  we  must  rely  on 
research  results  that  provide  a  lower  standard  of  proof  Soldiers  and  small  unit  leaders  who  have 
participated  in  virtual  simulation  exercises  have  consistently  reported  that  their  skills  have 
improved  as  a  result  of  the  training.  There  is  ample  evidence  that  Soldiers  report  that  their  skills 
have  improved  as  a  result  of  the  use  of  immersive  simulation,  but  there  is  little  objective 
evidence  (such  as  trainer  ratings  or  mission  success)  to  confirm  this,  or  to  show  how  the 
effectiveness  of  training  in  virtual  simulations  compares  with  other  approaches.  Those 
evaluations  that  have  been  conducted  have  also  generally  involved  fewer  Soldiers  and  units  than 
would  be  required  to  establish  statistical  significance. 

The  problem  is  that  there  has  not  been  an  organizational  commitment  to  conduct  such  an 
evaluation.  Largely  because  of  resource  constraints,  particularly  on  the  availability  of  Soldier 
trainees  to  participate  in  evaluations,  relatively  few  virtual  simulation  programs  for  dismounted 
Soldiers  have  evaluated  collective  training  effectiveness  using  objective  measures  of  Soldier 
performance.  For  the  same  reason,  virtual  simulation  has  not  generally  been  compared  to  other 
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forms  of  training.  Ideally,  we  would  like  to  have  more  rigorous  data.  Realistically,  this  data  does 
not  exist  for  other  training  methods,  either. 

In  addition  to  the  unresolved  questions  about  the  acquisition  of  skills  in  the  simulators, 
there  are  some  questions  about  the  transfer  of  what  is  learned  in  the  virtual  simulators  to  the  real 
world.  Pleban  and  Salvetti  (2003)  found  that  practicing  building  clearing  in  a  VE  did  not 
improve  performance  on  the  task  in  the  real  building.  Statkus  and  Spring  (2003)  found 
differences  in  Soldier  task  performance  between  MOUT  tasks  performed  in  virtual  simulation 
(SVS™)  and  performance  of  the  same  tasks  in  a  live  simulation  with  simunitions.  Soldiers  in 
the  SVS™  engaged  targets  more  rapidly  and  fired  fewer  rounds  than  did  Soldiers  conducting  the 
same  exercises  at  the  McKenna  MOUT  site.  They  attributed  the  differences  in  performance  to  a 
combination  of  environmental  effects  (lighting  and  temperature),  load  carried  (BDU  only  vs.  full 
fighting  load),  increased  fatigue  in  the  live  simulation  (as  a  result  of  the  previous  two  factors), 
and  the  adverse  consequence  of  being  shot  with  a  simunition. 

PC-based  simulations,  or  PC  games  modified  for  training  purposes,  are  receiving 
increasing  attention.  A  number  have  been  developed  or  modified  for  Army  training.  The 
general  evaluation  finding  is  that  trainees  respond  positively  to  the  game,  but  objective  measures 
of  training  effectiveness  are  not  obtained.  An  ARl  workshop  on  the  topic  concluded  that  “while 
most  researchers  seem  to  agree  that  computer  games  can  be  effective  training  tools,  there  is  very 
little  empirical  evidence  to  support  that  conclusion”  (Belanich,  Mullen,  and  Dressel,  2004). 

In  addition  to  the  question  of  effectiveness,  there  is  also  the  question  of  whether  potential 
Solider  trainees,  both  enlisted  and  junior  officer,  possess  the  computer  and  game-playing  skills 
that  would  enable  them  to  master  PC  games  used  for  training  readily.  There  are  no  wide-scale 
survey  data  on  this  question,  but  data  from  experiments  with  relatively  small  numbers  of 
participants  suggests  that  20-57%  of  junior  officers  and  7-22%  of  enlisted  personnel  do  not  play 
interactive  games.  This  indicates  a  need  for  introductory  practice  prior  to  use  of  interactive 
games  for  training,  but  the  existing  data  are  very  limited. 

Immersive  simulations  (or  PC-based  simulations)  should  not  be  used  by  themselves. 

They  can  provide  a  transition  between  initial  or  classroom  training  and  live  simulations.  They 
appear  to  be  best  suited  to  training  cognitive  skills  (not  motor  skills),  decision  making, 
communication,  and  teamwork.  They  require  an  instructor  to  provide  a  context  and  feedback. 

Desktop  or  laptop  simulators  should  be  considered  as  an  alternative  to  fully-immersive 
simulators  which  use  large  rear-projection  or  HMD.  Fully-immersive  head-tracked  displays  are 
more  effective,  particularly  when  performing  spatially-oriented  tasks  and  acquiring  spatial 
knowledge,  but  based  on  a  limited  number  of  experiments,  this  difference  does  not  appear  to  be 
large,  and  may  not  justify  the  increased  cost. 

Summary  of  Findings 

The  major  findings  are  summarized  in  the  following  sections,  organized  around  the  topics 
of  training  effectiveness.  Soldier  task  performance,  and  advantages  and  disadvantages  of 
immersive  virtual  simulations. 
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What  have  we  learned  about  training  effectiveness? 

The  following  are  based  on  data  and  observation  from  five  assessments  of  immersive 
simulation  systems  involving  45  squad  and  fire  team  leaders  and  94  squad  members  during  the 
period  1999  -  2005. 

First,  Soldiers  and  small  unit  leaders  report  that  their  skills  improve  as  a  result  of  training 
in  virtual  simulations.  They  report  the  most  improvement  in  eontrolling,  eoordinating, 
eommuni eating,  and  planning  (conducted  prior  to  use  of  the  simulators),  and  less  improvement 
in  the  mechanics  of  tasks  and  battle  drills. 

Second,  these  self-reports  by  Soldiers  have  generally,  if  informally,  been  confirmed  by 
observers.  In  one  of  the  assessments  objective  ratings  of  unit  performance  by  two  independent 
raters  were  obtained.  These  ratings  showed  improvement  in  performance  over  the  course  of  the 
training. 

These  results  are  consistent  with  findings  involving  collective  marksmanship  trainers. 
Using  different  technology,  but  functionally  similar  to  immersive  virtual  simulators,  these 
trainers  have  produced  objective  data  indicating  that  they  provide  effective  training  in  tactical 
skills  as  well  as  marksmanship. 

It  should  be  noted  that  these  results  were  obtained  despite  a  number  of  limitations.  The 
total  amount  of  time  that  trainees  spent  conducting  training  scenarios  was  actually  relatively 
short  (no  more  than  three  hours).  In  most  cases,  the  software  was  still  under  development,  and 
not  fully  problem  free.  The  scenarios  were  usually  not  tailored  to  the  skill  level  of  the  leader  or 
unit.  The  trainer/AAR  leader  was  frequently  unfamiliar  with  the  technology  and  did  not 
necessarily  use  it  to  best  advantage. 

What  we  have  learned  about  what  Soldier  task  performance? 

Fifty-one  Soldiers  and  leaders  participated  in  tactical  exercises  using  immersive 
simulators  (all  variants  of  the  immersive  Soldier  Visualization  Station,  or  SVS™)  to  conduct 
tactical  exercises  in  assessments  conducted  during  the  period  2002  -  2005.  While  simulation 
technology  in  general  and  the  SVS^*^  in  particular  were  evolving,  the  capabilities  were  fairly 
static  during  that  period.  After  Soldiers  participated  in  the  scenarios,  they  rated  how  well  they 
could  perform  each  of  54  Soldier  activities  in  the  simulators  as  Very  Poor,  Poor,  Good,  or  Very 
Good.  Overall,  30  of  the  54  activities  were  rated  Good  or  better;  18  were  rated  between  the 
Good/Poor  midpoint  and  Good;  and  6  were  rated  Poor.  Activities  which  were  rated  highly 
included  outdoor  movement,  identification  of  types  of  people  (civilians,  non-combatants  within  a 
room,  enemy  soldiers),  identification  of  tactically  significant  areas  (sectors  of  observation  and 
responsibility),  and  individual  weapons  use  (but  not  grenades).  Poorly  rated  items  included 
maneuver  indoors  (close  to  others,  past  furniture,  close  to  walls,  around  objects,  past  other 
personnel,  around  comers,  through  doorways,  up  and  down  stairs),  and  identifying  the  source 
and  type  of  fire  (enemy  or  friendly),  either  by  auditory  or  visual  cues. 
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The  issue  of  fine  or  precision  movement  in  confined  areas,  such  as  movement  indoors, 
has  consistently  emerged  as  the  most  important  improvements  required.  It  appears  to  be  a  more 
complicated  problem  than  can  be  solved  by  building  a  new  movement  control  device.  It  involves 
a  number  of  related  issues:  the  size  and  complexity  of  the  “bounding  box”  which  detects 
collisions  between  Soldiers  and  other  objects,  and  between  Soldiers;  the  limited  visual  texture 
and  shading  cues  on  the  walls  inside  buildings;  the  relatively  narrow  FOV  of  the  SVS™  display; 
the  problem  representing  objects  which  are  located  between  the  Soldier  and  the  rear-projection 
screen;  and  the  linkage  between  direction  of  gaze  and  direction  of  movement. 

Precise  movement  in  buildings  is  the  primary  example  of  the  limitation  on  the  physical 
actions  that  trainees  could  perform  in  the  SVS™  simulator.  Other  simulators  may  differ  in  the 
specifics  of  the  physical  actions  that  can  be  performed.  But  no  matter  what  the  design,  there  are 
likely  to  be  limitations,  particularly  with  regard  to  locomotion  and  touch.  For  example,  trainees 
in  the  SVS™  could  drop  to  the  ground,  but  they  could  not  drop  to  the  ground  and  roll.  If  they 
tried,  they  were  likely  to  get  tangled  up  in  wires.  They  could  not  physically  “throw”  grenades. 
They  selected  a  type  of  grenade  from  a  menu,  chose  direction  and  launch  angle  by  pointing  their 
mock  weapon,  choose  the  amount  of  force  desired  by  adjusting  a  slider  with  a  pair  of  buttons  on 
the  weapon,  and  pulled  the  trigger  to  launch  the  grenade.  While  this  bears  little  resemblance  to 
the  physical  act  of  throwing  a  real  grenade,  it  does  provide  practice  in  deciding  which  type  of 
grenade  to  use,  and  where  and  when  to  employ  it.  They  eould  not  physieally  toueh  other  people 
in  the  simulation,  or  objects  other  than  their  individual  weapon  and  equipment.  They 
eonsequently  could  not  practice  some  aspects  of  room  entry. 

These  simulator  eonstraints  limit  training  effectiveness  only  if  those  actions  that  cannot 
be  perfonued  in  the  simulator  are  not  trained  by  other  means.  For  example,  the  SVS™  approach 
to  grenade  throwing  is  a  problem  only  if  Soldiers  are  not  taught  to  throw  grenades  by  other 
means.  It  is  important  that  immersive  virtual  simulation  be  conducted  as  part  of  a  planned 
sequence  of  instruction  that  provides  them  with  the  prerequisite  skills  prior  to  the  simulation,  and 
provides  subsequent  live  training  to  improve  the  physical  skills  that  cannot  be  trained  in  the 
virtual  simulation. 

Even  though  the  simulator  may  not  allow  Soldiers  to  perfonu  many  of  the  physical 
actions  they  need  to  perform  in  the  way  they  need  to  perfonn  them,  it  does  allow  them  to 
practice  and  learn  the  cognitive  skills  they  need.  These  include  the  planning  that  needs  to  be 
done  prior  to  missions,  the  skills  required  to  maintain  situational  awareness,  and  learning  to  think 
on  their  feet  and  make  appropriate  decisions  in  complex  situations.  These  are  all  positive  things 
that  are  one  level  of  analysis  above  the  physical  actions  Soldiers  can  or  cannot  do  in  the  virtual 
environments  with  current  technology. 

What  are  the  advantages  of  immersive  virtual  simulations? 

Virtual  simulations  possess  a  number  of  advantages  relative  to  the  use  of  live  simulation 
for  dismounted  Soldier  training.  These  are  described  in  the  following  paragraphs. 

Virtual  simulation  ean  represent  any  terrain  or  environment  that  has  been  modeled, 
whether  a  real  location  or  a  generic  one.  It  is  not  limited  to  the  physical  area  where  it  is  located. 
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Therefore,  it  can  provide  an  urban  training  area  with  high  rise  buildings  at  a  remote  location,  or  a 
Middle  Eastern  village  in  a  downtown  armory.  Environments  can  be  selected  based  on  training 
needs,  rather  than  availability. 

Weather  and  other  environmental  conditions  (rain,  fog)  can  be  represented.  Nighttime 
missions  can  be  trained  during  the  day  and  daytime  missions  can  be  trained  during  the  night. 
There  is  no  need  to  wait  for  the  right  conditions. 

Use  of  immersive  virtual  simulation  is  likely  to  reduce  the  amount  of  time  required  by  a 
unit  to  conduct  a  training  exercise  relative  to  live  training.  The  following  requirements  should  be 
reduced:  transportation  to  and  from  the  training  site;  drawing,  cleaning,  and  returning  weapons, 
ammunition,  and  gear  such  as  MlLES/OneTESS;  positioning  equipment  and  support  personnel 
prior  to  the  exercise;  movement  to  the  exercise  start  position;  movement  to  the  AAR  area;  and 
cleaning  up  the  training  site. 

Both  instrumented  live  and  virtual  simulation  require  support  personnel,  and  both  require 
support  personnel  with  technical  skills  to  maintain  and  operate  the  equipment.  However,  the 
requirements  for  human  role  players  fill  the  positions  of  enemy  units,  other  friendly  units,  and 
civilians  should  be  much  smaller  for  virtual  simulations.  OneSAF  can  fill  many  of  these  roles 
(enemy  units,  crowds,  etc.).  Human  role  players  can  fill  multiple  roles  (although  not 
simultaneously)  without  regard  to  physical  appearance  or  location  in  the  environment. 

The  risk  of  accident  or  injury  is  lower  in  virtual  than  in  live  simulation.  Because  of  this, 
virtual  simulations  make  it  possible  to  more  realistically  depict  the  effects  of  demolitions  (blow  a 
hole  in  the  building  anywhere)  or  indirect  fire  than  is  possible  in  live  simulations. 

While  virtual  simulations  require  improved  physical  facilities,  they  require  less  physical 
space  than  a  live  simulation  facility.  They  also  reduce  the  need  for  use  of  actual  equipment. 

Virtual  AARs  provide  unique  capabilities  for  viewing  events  that  are  not  currently 
possible  with  live  simulations.  First,  they  can  provide  an  overall  view  of  the  action  that  is  not 
possible  with  video  cameras,  which  provide  multiple  separate  views  of  portions  of  the  action. 
Second,  they  can  provide  a  view  of  the  action  from  any  perspective,  not  limited  by  camera 
locations.  Third,  they  capture  and  provide  an  accurate  representation  of  ground  truth  on  which  to 
base  AARs  so  they  can  learn  from  their  mistakes.  Finally,  they  can  provide  a  variety  of  aids  to 
analyze  and  depict  events  (e.g.,  shot  lines,  movement  paths,  FOV,  etc.). 

These  capabilities  make  it  possible  for  virtual  immersive  simulations  to  be  used  to 
provide  the  opportunity  for  small  unit  leaders  and  unit  members  to  practice,  in  the  eontext  of 
realistic  simulated  operations,  the  cognitive  and  decision  making  skills  that  will  foster 
adaptability  and  the  capability  to  respond  to  rapidly  changing  situations.  Focused,  repetitive, 
deliberate  practice,  with  feedback  based  on  performance,  is  an  effective  method  for  training  the 
recognition  of  situations  and  developing  expertise.  Cognitive  and  decision  making  skills  can  be 
trained  even  if  some  physical  tasks  cannot  be  performed  in  the  situation. 
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What  are  the  disadvantages  of  immersive  virtual  simulations? 

Virtual  simulations  possess  a  number  of  disadvantages  relative  to  the  use  of  live 
simulation  for  dismounted  Soldier  training.  These  are  described  in  the  following  paragraphs. 

Some  physical  activities  cannot  be  performed  in  the  simulators,  or  must  be  performed 
differently  in  the  simulators  than  in  the  real  world.  These  currently  include  actions  such  as 
throwing  grenades,  or  touching  or  grasping  other  participants  or  objects.  There  is  a  risk  that 
Soldiers  may  learn  inappropriate  behaviors.  This  risk  can  be  reduced  by  having  trainers 
emphasize  the  differences  between  the  simulation  and  the  real  world,  and  by  providing  additional 
live  training  on  the  affected  tasks. 

Weapons  effects  may  not  be  modeled  correctly,  or  position  and  location  tracking  may  not 
be  accurate  enough  to  depict  realistic  weapons  effects.  Soldiers  may,  therefore,  develop 
inappropriate  expectations.  This  disadvantage  applies  to  live  simulation  as  well.  Any  specially 
developed  simulation  system  should  model  weapons  effects,  particularly  individual  Soldier 
weapons,  accurately.  Tracking  accuracy  may  be  a  problem  for  fully  immersive  systems,  and  if 
so,  would  suggest  that  training  scenarios  not  involve  long  range  use  of  individual  weapons  by 
trainees.  Inaccurate  modeling  may  be  a  problem  for  use  of  commercial  products. 

Virtual  simulations  require  improved  facilities  (enclosed  space,  air  conditioning,  and 
power).  They  also  require  technical  support  to  implement  training  scenarios  and  operate  and 
maintain  computers  and  other  equipment. 

Trainees  may  not  take  virtual  simulations  seriously  because  they  are  similar  to  video 
games,  and  might,  therefore,  practice  actions  that  are  inappropriate  in  the  real  world,  or  attempt 
to  “game”  the  system.  This  risk  can  be  reduced  by  trainer  emphasis. 

Some  trainees  may  experience  “simulator  sickness.”  This  is  unlikely  to  be  a  major 
problem.  Risks  can  be  reduced  by  making  initial  exposures  to  the  simulators  relatively  short,  and 
gradually  increasing  the  duration. 
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Recommendations 


Conduct  a  large-scale  evaluation  using  current  technology. 

As  discussed  above,  the  evidence  that  immersive  simulation  can  provide  effective 
training  is  not  sufficient  to  justify  the  immediate  acquisition  of  such  systems  on  a  large  scale. 
However,  there  is  enough  evidence  that  they  can  be  effective  to  justify  acquisition  of  a  prototype 
system  using  current  technology  and  use  it  to  conduct  a  rigorous  training  effectiveness 
evaluation  which  would  permit  quantification  of  the  effectiveness  of  the  training  and  comparison 
with  alternatives.  While  this  would  be  costly,  in  terms  of  both  dollars  and  Soldier  time,  it  would 
lead  to  a  more  informed  acquisition  decision. 

This  evaluation  should  consider  objective  measurement  of  both  skill  improvement  on  the 
simulator  and  transfer  of  those  skills  to  a  live  simulation  environment.  It  should  involve  a 
sufficient  number  of  units  (probably  squads)  that  any  meaningful  real  differences  in  the  training 
effectiveness  of  different  can  be  detected.  The  evaluation  should  be  embedded  in  the  unit’s 
normal  training  progression,  and  unit  personnel  should  be  involved  in  its  development  and 
delivery. 

Consider  cost-effectiveness  of  fully  immersive  and  desktop  or  laptop  systems. 

The  cost  differential  between  fully-immersive  simulators  using  large  projection  displays 
or  HMDs  and  simulators  using  desktop  or  laptop  computers  to  provide  the  same  functionality 
can  be  enormous.  Knerr  (2006),  analyzed  cost  data  provided  by  Loftin,  Dryer,  Belfore,  Petty, 
Phillips,  Garcia,  Seevi,  Lusso,  Mastaglio,  and  Park  (2004),  and  estimated  the  difference  at  over 
$75,000  per  simulator.  While  the  cost  difference  is  likely  to  decrease  as  the  cost  of  large  visual 
displays  decreases,  the  fact  remains  that  the  major  cost  drivers  for  fully  immersive  simulators  are 
the  interface  devices,  particularly  the  position  and  orientation  trackers  and  visual  display 
systems.  The  computers  are  relatively  cheap.  The  interface  devices  also  increase  space  and 
support  requirements. 

In  contrast,  the  research  evidence  indicates  that  any  difference  in  training  effectiveness  between 
immersive  and  desktop  systems  is  likely  to  be  small.  Immersive  systems  are  more  effective, 
particularly  when  training  spatially-oriented  tasks,  such  as  learning  a  complex  route  or  floor 
plan.  Whether  immersive  simulators  are  also  better  when  training  squads  to  conduct  urban  or 
counter-insurgency  operations,  and  if  so,  whether  the  difference  is  large  enough  to  justify  the 
increased  cost,  are  unknown,  but  should  be  investigated,  perhaps  in  conjunction  with  the  large- 
scale  evaluation  recommended  above. 

Conduct  a  survey  of  potential  officer  and  enlisted  trainees  to  determine  game-playing  skill. 

One  of  the  assumptions  underlying  the  growing  popularity  of  the  use  of  video  and  PC- 
based  games  for  training  is  that  the  potential  trainees,  if  not  avid  game  players,  are  at  least 
interested  in  and  experienced  with  such  games.  They,  therefore,  need  little  training  in  their  use, 
and  are  motivated  to  play.  However,  data  from  small  samples  of  Soldiers  who  participated  in 
evaluations  suggests  that  this  may  not  be  the  case.  While  a  fairly  large  percentage  of  male 


47 


enlisted  personnel,  possibly  75%  or  higher,  play  computer  or  video  games  weekly  or  more 
frequently,  officers  appear  to  play  less  frequently.  In  addition,  based  on  civilian  data,  we  would 
expect  that  female  Soldiers  would  play  less  frequently  than  male  Soldiers.  We  know  little  about 
the  types  of  games  that  Soldiers  play  (e.g.,  first  person  shooter  vs.  role  player).  Since  game 
playing  experience  has  implications  for  the  types  of  games  that  could  be  used  for  training  and  the 
amount  and  type  of  prerequisite  game  playing  training  required,  we  would  like  to  have  a  much 
better  picture  of  the  both  current  and  projected  Soldier  and  leader  familiarity  and  experience  with 
different  types  of  interactive  games. 

Continue  to  improve  virtual  simulation  capabilities  and  assess  their  training  effectiveness. 

Some  aspects  of  immersive  virtual  simulation  technology,  such  as  large-screen  displays, 
will  continue  to  improve,  or  become  less  expensive,  as  a  result  of  commercial  market  forces. 
However,  it  will  require  a  continuing  science  and  technology  investment  by  the  Army  to  produce 
a  tailored  system  that  provides  cost-effective  training. 
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Appendix  A 
List  of  Acronyms 


AAR 

After  Action  Review 

ARI 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences 

ARL 

U.S.  Army  Research  Laboratory 

ARTEP 

Army  Training  and  Evaluation  Program 

CATT 

Combined  Army  Tactical  Trainer 

CAVE 

CAVE  Automatic  Virtual  Environment 

CE 

Culminating  Event 

CPT 

Combat  Proficiency  Training 

COTS 

Commercial  off-the-Shelf 

DARPA 

Defense  Advanced  Projects  Agency 

DARWARS 

DARPA  Training  Superiority  Program 

DCSOPS&T 

Deputy  chief  of  Staff  for  Operations  and  Training 

DI 

Dismounted  Infantry 

DIS 

Distributed  Interactive  Simulation 

DISAF 

Dismounted  Infantry  Semi-Automated  Forces 

DIVAARS 

Dismounted  Infantry  Virtual  After  Action  Review  System 

DoD 

Department  of  Defense 

OTIC 

Defense  Technical  Information  Center 

DWN 

Dismounted  Warrior  Network 

OWN  ERT 

Dismounted  Warrior  Network  Enhancements  for  Restricted 
Terrain 

EST 

Engagement  Skills  Trainer 

FOV 

Field  of  View 

FSC 

Full  Spectrum  Command 

HLA 

High  Level  Architecture 

HMD 

Head  Mounted  Display  or  Helmet  Mounted  Display 

ICCC 

Infantry  Captains  Career  Course 

lOBC 

Infantry  Officer  Basic  Course 

I-Port 

Individual  Portal 

LED 

Light  Emitting  Diode 

LFX 

Live  Fire  Exercise 

L-V-C 

Live  -  Virtual  -  Constructive 

MILES 

Multiple  Integrated  Laser  Engagement  System 

MOUT 

Military  Operations  in  Urban  Terrain 

MPR 

Mission  Planning  and  Rehearsal 

MTP 

Mission  Training  Plan 

ODT 

Omni-Directional  Treadmill 

OEF 

Operation  Enduring  Freedom 

OIF 

Operation  Iraqi  Freedom 

One  SAF 

One  Semi-Automated  Forces 

OneTESS 

One  Tactical  Engagement  Simulation  System 

OPTEMPO 

Operational  Tempo 
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ORD 

Operational  Requirements  Doeument 

PEO  STRI 

Program  Executive  Office  for  Simulation,  Training,  and 
Instrumentation 

PC 

Personal  Computer 

R&D 

Research  and  Development 

RDECOM  STTC 

Research,  Development,  and  Engineering  Command  Simulation, 
and  Training  Technology  Center 

RED 

Reality  by  Design 

RDT 

Rapid  Decision  Trainer 

RSTA 

Reconnaissance,  Surveillance,  and  Target  Acquisition 

SASO 

Support  and  Sustainment  Operations 

SAVE 

Situational  Awareness  in  Virtual  Environments 

SEE 

Soldier  Eattle  Lab 

SETS 

Squad  Engagement  Training  System 

SME 

Subject  Matter  Expert 

SOF 

Special  Operations  Forces 

SSE 

Squad  Synthetic  Environment 

SSQ 

Simulator  Sickness  Questionnaire 

STO 

Science  and  Technology  Objective 

STRICOM 

US  Army  Simulation,  Training  and  Instrumentation  Command 

svs™ 

Soldier  Visualization  Station 

SVSD 

Desktop  SVS™  simulator 

SVSI 

Immersive  SVS™  simulator 

TOE 

Table  of  Organization  and  Equipment 

TPIO 

TRADOC  Program  Integration  Office 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

TTES 

Team  Tactical  Engagement  Simulator 

UO 

Urban  Operations 

USAARMC 

U.S.  Army  Armor  Center 

USAIC 

U.S.  Army  Infantry  Center 

USAIS 

U.S.  Army  Infantry  School 

USJFKSWCS 

U.S.  John  F.  Kennedy  Special  Warfare  Center  and  Sehool 

VE 

Virtual  Environment 

V-IMTS 

Virtual  Integrated  MOUT  Training  System 

Vies 

Virtual  Individual  Combatant  Simulator 

VR 

Virtual  Reality 
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